[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:34 dist-test-slave-t3q3 krb5kdc[27539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:34 dist-test-slave-t3q3 krb5kdc[27539](info): set up 2 sockets
Aug 12 06:55:34 dist-test-slave-t3q3 krb5kdc[27539](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 06:55:35.882968 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.475s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:55:36 dist-test-slave-t3q3 krb5kdc[27539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:55:36.088614 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:41917
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:41917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:36.474020 27560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:36.474391 27560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:36.474555 27560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:36.482115 27560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:36.482275 27560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:36.482396 27560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:36.482497 27560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:36.482586 27560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:36.492835 27560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:41917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:41917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:36.494262 27560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:36.495585 27560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:36.508245 27570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:36.508333 27574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:36.508383 27572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:36.509505 27560 server_base.cc:1034] running on GCE node
I20260812 06:55:36.510327 27560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:36.511754 27560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:36.513018 27560 hybrid_clock.cc:648] HybridClock initialized: now 1786517736512938 us; error 103 us; skew 500 ppm
I20260812 06:55:36.513381 27560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:36 dist-test-slave-t3q3 krb5kdc[27539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517736, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:36.516747 27560 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:55:36.518544 27560 webserver.cc:460] Webserver started at http://127.26.226.62:33519/ using document root <none> and password file <none>
I20260812 06:55:36.519447 27560 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:36.519582 27560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:36.519991 27560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:36.523512 27560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "44858e6ce4bc43cb85e4d20283be4722"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-t3q3"
server_key: "c24596c9af0da82404f41715d4e2e117"
server_key_iv: "087c468d5ff9393e8d3bfb6383178ed1"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.524386 27560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "44858e6ce4bc43cb85e4d20283be4722"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-t3q3"
server_key: "c24596c9af0da82404f41715d4e2e117"
server_key_iv: "087c468d5ff9393e8d3bfb6383178ed1"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.529688 27560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:55:36.533121 27583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.534386 27560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:55:36.534577 27560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "44858e6ce4bc43cb85e4d20283be4722"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-t3q3"
server_key: "c24596c9af0da82404f41715d4e2e117"
server_key_iv: "087c468d5ff9393e8d3bfb6383178ed1"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.534792 27560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:36.559617 27560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:36.563936 27560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:36.564285 27560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:36.583724 27560 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:41917
I20260812 06:55:36.583763 27662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:41917 every 8 connection(s)
I20260812 06:55:36.585124 27560 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:36.588862 27665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:36.589749 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 27560
I20260812 06:55:36.589993 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:55:36.591770 27528 external_mini_cluster.cc:1428] Setting key e86fbce38527820e2ede3d3ffec8cb3d
I20260812 06:55:36.599753 27665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722: Bootstrap starting.
I20260812 06:55:36.602641 27665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:36.603629 27665 log.cc:824] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:36.606487 27665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722: No bootstrap required, opened a new log
I20260812 06:55:36.613761 27665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44858e6ce4bc43cb85e4d20283be4722" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41917 } }
I20260812 06:55:36.614151 27665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:36.614271 27665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44858e6ce4bc43cb85e4d20283be4722, State: Initialized, Role: FOLLOWER
I20260812 06:55:36.614948 27665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44858e6ce4bc43cb85e4d20283be4722" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41917 } }
I20260812 06:55:36.615211 27665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:36.615358 27665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:36.615552 27665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:36.617817 27665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44858e6ce4bc43cb85e4d20283be4722" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41917 } }
I20260812 06:55:36.618333 27665 leader_election.cc:302] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44858e6ce4bc43cb85e4d20283be4722; no voters:
I20260812 06:55:36.619004 27665 leader_election.cc:288] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:36.619110 27675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [term 1 FOLLOWER]: Leader election won for term 1
Aug 12 06:55:36 dist-test-slave-t3q3 krb5kdc[27539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:36.621747 27665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:36.622128 27675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [term 1 LEADER]: Becoming Leader. State: Replica: 44858e6ce4bc43cb85e4d20283be4722, State: Running, Role: LEADER
I20260812 06:55:36.622821 27675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44858e6ce4bc43cb85e4d20283be4722" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41917 } }
I20260812 06:55:36.627327 27678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44858e6ce4bc43cb85e4d20283be4722" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44858e6ce4bc43cb85e4d20283be4722" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41917 } } }
I20260812 06:55:36.627516 27679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44858e6ce4bc43cb85e4d20283be4722. Latest consensus state: current_term: 1 leader_uuid: "44858e6ce4bc43cb85e4d20283be4722" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44858e6ce4bc43cb85e4d20283be4722" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41917 } } }
I20260812 06:55:36.627867 27678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:36.627871 27679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:36.635731 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:36.595500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39168 (local address 127.26.226.62:41917)
0812 06:55:36.596338 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:36.596349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:36.597540 (+ 1191us) server_negotiation.cc:407] Connection header received
0812 06:55:36.606394 (+ 8854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:36.606414 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:36.606822 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:36.607230 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:36.610296 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.611670 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.613467 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.614204 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.614429 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.621219 (+ 6790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:36.621227 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:36.621242 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:36.621280 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:36.625603 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:36.626795 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:36.626805 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:36.626816 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:36.627122 (+ 306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:36.627798 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:36.627803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:36.627806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:36.628940 (+ 1134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:36.629468 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:36.633310 (+ 3842us) server_negotiation.cc:299] Negotiation successful
0812 06:55:36.633885 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":634,"thread_start_us":384,"threads_started":1}
I20260812 06:55:36.640918 27695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:36.646242 27695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:36.655222 27695 catalog_manager.cc:1347] Generated new cluster ID: b9fe4a27bb0445f19860acb908a7f28c
I20260812 06:55:36.655354 27695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:36.672823 27695 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:36.674891 27695 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:36.694358 27695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44858e6ce4bc43cb85e4d20283be4722: Generated new TSK 0
I20260812 06:55:36.695266 27695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:55:36.752311 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:41917
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:37.095465 27714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:37.095746 27714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:37.095846 27714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:37.103036 27714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:37.103142 27714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:37.103260 27714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:55:37.112913 27714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:41917
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:37.114174 27714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:37.115451 27714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:37.129576 27720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:37.129456 27721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:37.129485 27727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:37.130606 27714 server_base.cc:1034] running on GCE node
I20260812 06:55:37.131283 27714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:37.132285 27714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:37.133553 27714 hybrid_clock.cc:648] HybridClock initialized: now 1786517737133490 us; error 59 us; skew 500 ppm
I20260812 06:55:37.133931 27714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:37 dist-test-slave-t3q3 krb5kdc[27539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517737, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:37.137099 27714 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:55:37.138691 27714 webserver.cc:460] Webserver started at http://127.26.226.1:38473/ using document root <none> and password file <none>
I20260812 06:55:37.139394 27714 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:37.139585 27714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:37.139945 27714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:37.142899 27714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "5b2bd96f323c45bc85e0afa5a2131620"
format_stamp: "Formatted at 2026-08-12 06:55:37 on dist-test-slave-t3q3"
server_key: "ee3bf0cc06c39336895a8d10850ff727"
server_key_iv: "29d954dc9f601a61e4cec58da0584109"
server_key_version: "encryptionkey@0"
I20260812 06:55:37.143680 27714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "5b2bd96f323c45bc85e0afa5a2131620"
format_stamp: "Formatted at 2026-08-12 06:55:37 on dist-test-slave-t3q3"
server_key: "ee3bf0cc06c39336895a8d10850ff727"
server_key_iv: "29d954dc9f601a61e4cec58da0584109"
server_key_version: "encryptionkey@0"
I20260812 06:55:37.148958 27714 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 06:55:37.152493 27735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.153793 27714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 06:55:37.153963 27714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "5b2bd96f323c45bc85e0afa5a2131620"
format_stamp: "Formatted at 2026-08-12 06:55:37 on dist-test-slave-t3q3"
server_key: "ee3bf0cc06c39336895a8d10850ff727"
server_key_iv: "29d954dc9f601a61e4cec58da0584109"
server_key_version: "encryptionkey@0"
I20260812 06:55:37.154160 27714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:37.178977 27714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:37.183334 27714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:37.183667 27714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:37.184762 27714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:37.186439 27714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:37.186542 27714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.186681 27714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:37.186745 27714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.223469 27714 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:41913
I20260812 06:55:37.223613 27949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:41913 every 8 connection(s)
I20260812 06:55:37.224857 27714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:37.233070 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 27714
I20260812 06:55:37.233325 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:37.234134 27528 external_mini_cluster.cc:1428] Setting key c411dae62ce9b91ca370a73aaf25dd0d
Aug 12 06:55:37 dist-test-slave-t3q3 krb5kdc[27539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517737, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:37.249248 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.227636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:36223 (local address 127.26.226.62:41917)
0812 06:55:37.228290 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:37.228295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:37.229150 (+ 855us) server_negotiation.cc:407] Connection header received
0812 06:55:37.231220 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:37.231227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:37.231328 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:37.231462 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:37.233302 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.234467 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.236809 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.237503 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.237662 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.241775 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:37.241784 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:37.241791 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:37.241837 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:37.245640 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:37.246401 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:37.246410 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:37.246417 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:37.246590 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:37.247027 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:37.247035 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:37.247043 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:37.247463 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:37.247657 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:37.248401 (+ 744us) server_negotiation.cc:299] Negotiation successful
0812 06:55:37.248677 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":304,"threads_started":1}
I20260812 06:55:37.249795 27954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.228174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:41917 (local address 127.26.226.1:36223)
0812 06:55:37.228974 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:37.229016 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:37.230911 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:37.231733 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:37.231742 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:37.232419 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:37.233059 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.233079 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.234620 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.234624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.236644 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.236659 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.237738 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.237744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.238014 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.238862 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:37.238887 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:37.241528 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:37.245798 (+ 4270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:37.245805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:37.245817 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:37.246249 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:37.246714 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:37.246717 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:37.246720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:37.246899 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:37.247603 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:37.247608 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:37.247920 (+ 312us) client_negotiation.cc:769] Sending connection context
0812 06:55:37.248207 (+ 287us) client_negotiation.cc:240] Negotiation successful
0812 06:55:37.248596 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":324,"threads_started":1}
I20260812 06:55:37.251519 27950 heartbeater.cc:347] Connected to a master server at 127.26.226.62:41917
I20260812 06:55:37.251927 27950 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:37.253024 27950 heartbeater.cc:511] Master 127.26.226.62:41917 requested a full tablet report, sending...
I20260812 06:55:37.255477 27611 ts_manager.cc:194] Registered new tserver with Master: 5b2bd96f323c45bc85e0afa5a2131620 (127.26.226.1:41913)
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
I20260812 06:55:37.257033 27611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:36223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:55:37.287109 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:41917
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:37.626099 27958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:37.626452 27958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:37.626628 27958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:37.634043 27958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:37.634213 27958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:37.634410 27958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:55:37.644351 27958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:41917
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:37.645732 27958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:37.647071 27958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:37.660317 27965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:37.660392 27968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:37.660431 27966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:37.661837 27958 server_base.cc:1034] running on GCE node
I20260812 06:55:37.662516 27958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:37.663504 27958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:37.664748 27958 hybrid_clock.cc:648] HybridClock initialized: now 1786517737664710 us; error 57 us; skew 500 ppm
I20260812 06:55:37.665088 27958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:37 dist-test-slave-t3q3 krb5kdc[27539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517737, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:37.668345 27958 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:55:37.669874 27958 webserver.cc:460] Webserver started at http://127.26.226.2:41189/ using document root <none> and password file <none>
I20260812 06:55:37.670588 27958 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:37.670739 27958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:37.671065 27958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:37.674137 27958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "389db0f9633c4f9ab28d9b7263c0c490"
format_stamp: "Formatted at 2026-08-12 06:55:37 on dist-test-slave-t3q3"
server_key: "7debfee28595c642a52e44c0ffdc3a49"
server_key_iv: "8ca135056fc8c59ddf3ff4033c847352"
server_key_version: "encryptionkey@0"
I20260812 06:55:37.674943 27958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "389db0f9633c4f9ab28d9b7263c0c490"
format_stamp: "Formatted at 2026-08-12 06:55:37 on dist-test-slave-t3q3"
server_key: "7debfee28595c642a52e44c0ffdc3a49"
server_key_iv: "8ca135056fc8c59ddf3ff4033c847352"
server_key_version: "encryptionkey@0"
I20260812 06:55:37.680263 27958 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 06:55:37.683836 27978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.685349 27958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:37.685547 27958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "389db0f9633c4f9ab28d9b7263c0c490"
format_stamp: "Formatted at 2026-08-12 06:55:37 on dist-test-slave-t3q3"
server_key: "7debfee28595c642a52e44c0ffdc3a49"
server_key_iv: "8ca135056fc8c59ddf3ff4033c847352"
server_key_version: "encryptionkey@0"
I20260812 06:55:37.685763 27958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:37.710043 27958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:37.714578 27958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:37.714911 27958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:37.716090 27958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:37.717546 27958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:37.717656 27958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.717813 27958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:37.717888 27958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.753293 27958 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:42169
I20260812 06:55:37.753391 28176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:42169 every 8 connection(s)
I20260812 06:55:37.754660 27958 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:37.757592 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 27958
I20260812 06:55:37.757835 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:37.758352 27528 external_mini_cluster.cc:1428] Setting key 57c1d4c8afbfec688f046eead5f61063
Aug 12 06:55:37 dist-test-slave-t3q3 krb5kdc[27539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517737, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:37.779523 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.757548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:41677 (local address 127.26.226.62:41917)
0812 06:55:37.758194 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:37.758200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:37.759433 (+ 1233us) server_negotiation.cc:407] Connection header received
0812 06:55:37.762005 (+ 2572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:37.762009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:37.762075 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:37.762171 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:37.764594 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.765364 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.767937 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.768741 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.768907 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.773034 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:37.773039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:37.773043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:37.773080 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:37.775958 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:37.777059 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:37.777063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:37.777065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:37.777216 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:37.777761 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:37.777765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:37.777767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:37.778024 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:37.778135 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:37.779077 (+ 942us) server_negotiation.cc:299] Negotiation successful
0812 06:55:37.779254 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":313,"threads_started":1}
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
I20260812 06:55:37.781036 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.758179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:41917 (local address 127.26.226.2:41677)
0812 06:55:37.759230 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:37.759274 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:37.761676 (+ 2402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:37.762485 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:37.762501 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:37.763407 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:37.764386 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.764416 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.765529 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.765538 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.767725 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.767745 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.769005 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.769011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.769237 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.770004 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:37.770030 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:37.772756 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:37.776138 (+ 3382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:37.776151 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:37.776172 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:37.776909 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:37.777340 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:37.777347 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:37.777353 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:37.777642 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:37.778164 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:37.778172 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:37.778661 (+ 489us) client_negotiation.cc:769] Sending connection context
0812 06:55:37.778993 (+ 332us) client_negotiation.cc:240] Negotiation successful
0812 06:55:37.779619 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":358,"threads_started":1}
I20260812 06:55:37.782495 28177 heartbeater.cc:347] Connected to a master server at 127.26.226.62:41917
I20260812 06:55:37.782914 28177 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
I20260812 06:55:37.783849 28177 heartbeater.cc:511] Master 127.26.226.62:41917 requested a full tablet report, sending...
I20260812 06:55:37.785583 27611 ts_manager.cc:194] Registered new tserver with Master: 389db0f9633c4f9ab28d9b7263c0c490 (127.26.226.2:42169)
I20260812 06:55:37.786419 27611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:41677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:55:37.807535 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:41917
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:38.146895 28185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:38.147192 28185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:38.147382 28185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:38.154736 28185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:38.154853 28185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:38.155071 28185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:55:38.165632 28185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:41917
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:38.167021 28185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:38.168421 28185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:38.182289 28193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:38.182245 28195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:38.182206 28198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:38.183213 28185 server_base.cc:1034] running on GCE node
I20260812 06:55:38.183863 28185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:38.184898 28185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:38.186127 28185 hybrid_clock.cc:648] HybridClock initialized: now 1786517738186071 us; error 43 us; skew 500 ppm
I20260812 06:55:38.186506 28185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:38 dist-test-slave-t3q3 krb5kdc[27539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517738, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:38.190227 28185 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:55:38.191936 28185 webserver.cc:460] Webserver started at http://127.26.226.3:36745/ using document root <none> and password file <none>
I20260812 06:55:38.192775 28185 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:38.192910 28185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:38.193274 28185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:38.196465 28185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "c268e12c60f141b3ade0d7ee8aedc9b8"
format_stamp: "Formatted at 2026-08-12 06:55:38 on dist-test-slave-t3q3"
server_key: "e2f90004cd3e920cedf8404f4eb60065"
server_key_iv: "ac9e3da0d2b0d0a5e0fdb5e9bb7b26c0"
server_key_version: "encryptionkey@0"
I20260812 06:55:38.197294 28185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "c268e12c60f141b3ade0d7ee8aedc9b8"
format_stamp: "Formatted at 2026-08-12 06:55:38 on dist-test-slave-t3q3"
server_key: "e2f90004cd3e920cedf8404f4eb60065"
server_key_iv: "ac9e3da0d2b0d0a5e0fdb5e9bb7b26c0"
server_key_version: "encryptionkey@0"
I20260812 06:55:38.203092 28185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 06:55:38.207264 28207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:38.208845 28185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:38.209050 28185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "c268e12c60f141b3ade0d7ee8aedc9b8"
format_stamp: "Formatted at 2026-08-12 06:55:38 on dist-test-slave-t3q3"
server_key: "e2f90004cd3e920cedf8404f4eb60065"
server_key_iv: "ac9e3da0d2b0d0a5e0fdb5e9bb7b26c0"
server_key_version: "encryptionkey@0"
I20260812 06:55:38.209297 28185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:38.233455 28185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:38.238139 28185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:38.238445 28185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:38.239494 28185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:38.240895 28185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:38.241004 28185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:38.241153 28185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:38.241238 28185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:38.260648 27950 heartbeater.cc:503] Master 127.26.226.62:41917 was elected leader, sending a full tablet report...
I20260812 06:55:38.276561 28185 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:44545
I20260812 06:55:38.276620 28399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:44545 every 8 connection(s)
I20260812 06:55:38.277880 28185 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:38.278432 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 28185
I20260812 06:55:38.278642 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:38.279258 27528 external_mini_cluster.cc:1428] Setting key c8d32a2ee714b826c7d26a65649c2a4f
Aug 12 06:55:38 dist-test-slave-t3q3 krb5kdc[27539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517738, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:38.297845 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:38.280672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:43705 (local address 127.26.226.62:41917)
0812 06:55:38.281204 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:38.281212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:38.282595 (+ 1383us) server_negotiation.cc:407] Connection header received
0812 06:55:38.284491 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:38.284495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:38.284572 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:38.284665 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:38.286118 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.286880 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.288473 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.288857 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.288955 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.292489 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:38.292494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:38.292497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:38.292529 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:38.295206 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.295889 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.295893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.295896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.296017 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.296462 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.296465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.296467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.296723 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:38.296837 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:38.297422 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 06:55:38.297577 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":256,"threads_started":1}
I20260812 06:55:38.298707 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:38.281333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:41917 (local address 127.26.226.3:43705)
0812 06:55:38.282426 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:38.282472 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:38.284251 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:38.284877 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:38.284885 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:38.285415 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:38.285971 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:38.285988 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.287032 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.287036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:38.288338 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:38.288351 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.289038 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.289043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:38.289300 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.289981 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:38.290003 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:38.292300 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:38.295319 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:38.295330 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:38.295343 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:38.295771 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:38.296179 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:38.296183 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:38.296185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:38.296358 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:38.296831 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:38.296836 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:38.297104 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 06:55:38.297318 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 06:55:38.297657 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":614,"threads_started":1}
I20260812 06:55:38.300036 28400 heartbeater.cc:347] Connected to a master server at 127.26.226.62:41917
I20260812 06:55:38.300376 28400 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:38.301349 28400 heartbeater.cc:511] Master 127.26.226.62:41917 requested a full tablet report, sending...
I20260812 06:55:38.302848 27611 ts_manager.cc:194] Registered new tserver with Master: c268e12c60f141b3ade0d7ee8aedc9b8 (127.26.226.3:44545)
I20260812 06:55:38.303628 27611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:43705
I20260812 06:55:38.307493 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:55:38 dist-test-slave-t3q3 krb5kdc[27539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517738, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:55:38 dist-test-slave-t3q3 krb5kdc[27539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517738, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:55:38.371487 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:38.345948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56168 (local address 127.26.226.1:41913)
0812 06:55:38.347400 (+ 1452us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:38.347411 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:38.347642 (+ 231us) server_negotiation.cc:407] Connection header received
0812 06:55:38.347945 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:38.347953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:38.348241 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:38.348472 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:38.351160 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.352989 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.355097 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.356382 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.356545 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.363103 (+ 6558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:38.363116 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:38.363132 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:38.363191 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:38.367511 (+ 4320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.368503 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.368511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.368514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.368727 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.369237 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.369244 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.369251 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.369693 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:38.370136 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:38.370492 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 06:55:38.370884 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1142,"thread_start_us":757,"threads_started":1}
W20260812 06:55:38.373026 27796 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56168
Aug 12 06:55:38 dist-test-slave-t3q3 krb5kdc[27539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:55:38 dist-test-slave-t3q3 krb5kdc[27539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:55:38.429307 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:38.410761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56176 (local address 127.26.226.1:41913)
0812 06:55:38.411009 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:38.411015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:38.411269 (+ 254us) server_negotiation.cc:407] Connection header received
0812 06:55:38.411497 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:38.411502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:38.411664 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:38.411787 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:38.413416 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.414511 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.415808 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.416619 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.416766 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.422535 (+ 5769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:38.422540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:38.422544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:38.422580 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:38.426299 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.426939 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.426943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.426946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.427136 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.427524 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.427528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.427530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.427928 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:38.428276 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:38.428600 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 06:55:38.428832 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 06:55:38.431218 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 27714
I20260812 06:55:38.449616 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:38.432797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.26.226.1:41913)
0812 06:55:38.433031 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:38.433039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:38.433059 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:38.433261 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:38.433265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:38.433321 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:38.433428 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:38.435344 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.436052 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.437986 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.438800 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.438942 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.442415 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:38.442420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:38.442424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:38.442453 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:38.446179 (+ 3726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.447000 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.447008 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.447015 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.447252 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.447735 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.447742 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.447745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.448177 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:38.448521 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:38.448808 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 06:55:38.449054 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 06:55:38.450798 27937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:38.539165 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 27714
I20260812 06:55:38.566067 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 27958
Aug 12 06:55:38 dist-test-slave-t3q3 krb5kdc[27539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:55:38.582930 28418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:38.567044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.26.226.2:42169)
0812 06:55:38.567804 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:38.567812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:38.567838 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:55:38.567900 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:38.567906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:38.568211 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:38.568343 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:38.569523 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.570589 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.571775 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.572448 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.572544 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.577479 (+ 4935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:38.577489 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:38.577504 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:38.577542 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:38.580353 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.580983 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.580990 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.580992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.581118 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.581486 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.581489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.581491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.581837 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:38.582084 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:38.582323 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 06:55:38.582520 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":456,"threads_started":1}
I20260812 06:55:38.584344 28169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:38.669554 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 27958
I20260812 06:55:38.697255 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 28185
Aug 12 06:55:38 dist-test-slave-t3q3 krb5kdc[27539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:55:38.713919 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:38.698438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55704 (local address 127.26.226.3:44545)
0812 06:55:38.699139 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:38.699146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:38.699177 (+ 31us) server_negotiation.cc:407] Connection header received
0812 06:55:38.699244 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:38.699251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:38.699468 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:38.699602 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:38.700940 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.701967 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.703228 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.703792 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.703889 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.708366 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:38.708377 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:38.708392 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:38.708432 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:38.711139 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.711787 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.711794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.711796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.711926 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:38.712344 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:38.712348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:38.712349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:38.712711 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:38.712937 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:38.713331 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 06:55:38.713593 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":377,"threads_started":1}
I20260812 06:55:38.715262 28392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:38.802806 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 28185
I20260812 06:55:38.830709 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 27560
I20260812 06:55:38.831909 27655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:38.936517 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 27560
2026-08-12T06:55:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4606 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:39 dist-test-slave-t3q3 krb5kdc[28431](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:39 dist-test-slave-t3q3 krb5kdc[28431](info): set up 2 sockets
Aug 12 06:55:39 dist-test-slave-t3q3 krb5kdc[28431](info): commencing operation
krb5kdc: starting...
W20260812 06:55:40.257831 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:55:40 dist-test-slave-t3q3 krb5kdc[28431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:55:40.474026 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33385
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:33489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:33385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:40.820387 28448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:40.820701 28448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:40.820796 28448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:40.828393 28448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:40.828516 28448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:40.828568 28448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:40.828627 28448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:40.828672 28448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:40.839035 28448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:33489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:33385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:40.840518 28448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:40.841796 28448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:40.854352 28456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:40.854331 28462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:40.854442 28457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:40.857546 28448 server_base.cc:1034] running on GCE node
I20260812 06:55:40.858349 28448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:40.859767 28448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:40.860980 28448 hybrid_clock.cc:648] HybridClock initialized: now 1786517740860914 us; error 50 us; skew 500 ppm
I20260812 06:55:40.861351 28448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:40 dist-test-slave-t3q3 krb5kdc[28431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517740, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:40.868465 28448 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:55:40.869972 28448 webserver.cc:460] Webserver started at http://127.26.226.62:44481/ using document root <none> and password file <none>
I20260812 06:55:40.870642 28448 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:40.870759 28448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:40.871039 28448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:40.874106 28448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "e81c08431b0a439ba6d8883a7cea9e76"
format_stamp: "Formatted at 2026-08-12 06:55:40 on dist-test-slave-t3q3"
server_key: "1daea5652803d4bb82b32bf654320871"
server_key_iv: "68b12ebf470948a981508b6b904a2b6d"
server_key_version: "encryptionkey@0"
I20260812 06:55:40.874825 28448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "e81c08431b0a439ba6d8883a7cea9e76"
format_stamp: "Formatted at 2026-08-12 06:55:40 on dist-test-slave-t3q3"
server_key: "1daea5652803d4bb82b32bf654320871"
server_key_iv: "68b12ebf470948a981508b6b904a2b6d"
server_key_version: "encryptionkey@0"
I20260812 06:55:40.879940 28448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 06:55:40.883354 28473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:40.884737 28448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:40.884938 28448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "e81c08431b0a439ba6d8883a7cea9e76"
format_stamp: "Formatted at 2026-08-12 06:55:40 on dist-test-slave-t3q3"
server_key: "1daea5652803d4bb82b32bf654320871"
server_key_iv: "68b12ebf470948a981508b6b904a2b6d"
server_key_version: "encryptionkey@0"
I20260812 06:55:40.885156 28448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:40.897181 28448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:40.901206 28448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:40.901513 28448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:40.920673 28448 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:33385
I20260812 06:55:40.920702 28563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:33385 every 8 connection(s)
I20260812 06:55:40.922171 28448 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:40.925669 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 28448
I20260812 06:55:40.925952 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:55:40.926442 28564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:40.926858 27528 external_mini_cluster.cc:1428] Setting key 37848f4f0229fe91a89901dc7e18225b
I20260812 06:55:40.941802 28564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76: Bootstrap starting.
Aug 12 06:55:40 dist-test-slave-t3q3 krb5kdc[28431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:40.945199 28564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:40.946280 28564 log.cc:824] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:40.949290 28564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76: No bootstrap required, opened a new log
I20260812 06:55:40.950788 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:40.929112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51646 (local address 127.26.226.62:33385)
0812 06:55:40.929974 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:40.929986 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:40.930030 (+ 44us) server_negotiation.cc:407] Connection header received
0812 06:55:40.931687 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:40.931708 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:40.932179 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:40.932516 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:40.934421 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.935691 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.937473 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.938232 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.938373 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:40.943278 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:40.943288 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:40.943307 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:40.943347 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:40.946636 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:40.947353 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:40.947363 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:40.947369 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:40.947537 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:40.947946 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:40.947950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:40.947952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:40.948551 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:40.948871 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:40.949392 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 06:55:40.949810 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":420,"threads_started":1}
I20260812 06:55:40.957238 28564 raft_consensus.cc:348] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c08431b0a439ba6d8883a7cea9e76" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33385 } }
I20260812 06:55:40.957620 28564 raft_consensus.cc:374] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:40.957762 28564 raft_consensus.cc:729] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e81c08431b0a439ba6d8883a7cea9e76, State: Initialized, Role: FOLLOWER
I20260812 06:55:40.958392 28564 consensus_queue.cc:260] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c08431b0a439ba6d8883a7cea9e76" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33385 } }
I20260812 06:55:40.958637 28564 raft_consensus.cc:388] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:40.958792 28564 raft_consensus.cc:482] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:40.958982 28564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:40.961536 28564 raft_consensus.cc:504] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c08431b0a439ba6d8883a7cea9e76" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33385 } }
I20260812 06:55:40.962051 28564 leader_election.cc:302] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e81c08431b0a439ba6d8883a7cea9e76; no voters:
I20260812 06:55:40.962673 28564 leader_election.cc:288] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:40.962831 28574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:40.964603 28574 raft_consensus.cc:686] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [term 1 LEADER]: Becoming Leader. State: Replica: e81c08431b0a439ba6d8883a7cea9e76, State: Running, Role: LEADER
I20260812 06:55:40.964983 28564 sys_catalog.cc:564] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:40.965277 28574 consensus_queue.cc:237] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c08431b0a439ba6d8883a7cea9e76" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33385 } }
I20260812 06:55:40.968704 28575 sys_catalog.cc:455] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e81c08431b0a439ba6d8883a7cea9e76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c08431b0a439ba6d8883a7cea9e76" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33385 } } }
I20260812 06:55:40.968672 28577 sys_catalog.cc:455] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e81c08431b0a439ba6d8883a7cea9e76. Latest consensus state: current_term: 1 leader_uuid: "e81c08431b0a439ba6d8883a7cea9e76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c08431b0a439ba6d8883a7cea9e76" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33385 } } }
I20260812 06:55:40.969157 28575 sys_catalog.cc:458] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:40.969184 28577 sys_catalog.cc:458] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:40.972831 28584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:40.978642 28584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:40.987980 28584 catalog_manager.cc:1347] Generated new cluster ID: 751bad1d42094034a1744f4d0819103b
I20260812 06:55:40.988153 28584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:41.023188 28584 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:41.024694 28584 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:41.039947 28584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e81c08431b0a439ba6d8883a7cea9e76: Generated new TSK 0
I20260812 06:55:41.041028 28584 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:55:41.099207 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33385
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:33489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:41.469317 28607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:41.469617 28607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:41.469728 28607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:41.477854 28607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:41.477981 28607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:41.478124 28607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:55:41.488308 28607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:33489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33385
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:41.489920 28607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:41.491361 28607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:41.505724 28621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:41.505769 28619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:41.505635 28616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:41.506971 28607 server_base.cc:1034] running on GCE node
I20260812 06:55:41.507663 28607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:41.508723 28607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:41.509959 28607 hybrid_clock.cc:648] HybridClock initialized: now 1786517741509917 us; error 54 us; skew 500 ppm
I20260812 06:55:41.510320 28607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:41 dist-test-slave-t3q3 krb5kdc[28431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517741, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:41.513777 28607 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:55:41.515313 28607 webserver.cc:460] Webserver started at http://127.26.226.1:38947/ using document root <none> and password file <none>
I20260812 06:55:41.516034 28607 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:41.516245 28607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:41.516577 28607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:41.519591 28607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "e864cc0d340848d285b8ca506eaf51b8"
format_stamp: "Formatted at 2026-08-12 06:55:41 on dist-test-slave-t3q3"
server_key: "9cbeddcd8f9ee17d0e65b5dc01401efb"
server_key_iv: "90fd2c76066560102630868b40180aee"
server_key_version: "encryptionkey@0"
I20260812 06:55:41.520468 28607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "e864cc0d340848d285b8ca506eaf51b8"
format_stamp: "Formatted at 2026-08-12 06:55:41 on dist-test-slave-t3q3"
server_key: "9cbeddcd8f9ee17d0e65b5dc01401efb"
server_key_iv: "90fd2c76066560102630868b40180aee"
server_key_version: "encryptionkey@0"
I20260812 06:55:41.525859 28607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:55:41.529433 28636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:41.530722 28607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:55:41.530915 28607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "e864cc0d340848d285b8ca506eaf51b8"
format_stamp: "Formatted at 2026-08-12 06:55:41 on dist-test-slave-t3q3"
server_key: "9cbeddcd8f9ee17d0e65b5dc01401efb"
server_key_iv: "90fd2c76066560102630868b40180aee"
server_key_version: "encryptionkey@0"
I20260812 06:55:41.531131 28607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:41.566897 28607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:41.571818 28607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:41.572171 28607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:41.573303 28607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:41.574838 28607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:41.574946 28607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:41.575106 28607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:41.575191 28607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:41.609651 28607 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:36529
I20260812 06:55:41.609715 28822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:36529 every 8 connection(s)
I20260812 06:55:41.610987 28607 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:41.615403 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 28607
I20260812 06:55:41.615643 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:41.616521 27528 external_mini_cluster.cc:1428] Setting key b694f7e7a5b4cb57244f9ff62b6a34d1
Aug 12 06:55:41 dist-test-slave-t3q3 krb5kdc[28431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517741, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:41.635695 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:41.613663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:49105 (local address 127.26.226.62:33385)
0812 06:55:41.614101 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:41.614106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:41.615284 (+ 1178us) server_negotiation.cc:407] Connection header received
0812 06:55:41.617795 (+ 2511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:41.617800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:41.617882 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:41.617978 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:41.620101 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.620838 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.623408 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.623802 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.623901 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:41.628012 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:41.628021 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:41.628029 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:41.628098 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:41.631907 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:41.632769 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:41.632779 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:41.632786 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:41.632986 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:41.633437 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:41.633445 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:41.633451 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:41.633876 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:41.634041 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:41.634873 (+ 832us) server_negotiation.cc:299] Negotiation successful
0812 06:55:41.635172 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":218,"threads_started":1}
I20260812 06:55:41.636631 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:41.614277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33385 (local address 127.26.226.1:49105)
0812 06:55:41.615052 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:41.615112 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:41.617494 (+ 2382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:41.618277 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:41.618288 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:41.619005 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:41.619866 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:41.619896 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.621022 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.621030 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:41.623246 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:41.623265 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.624043 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.624053 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:41.624424 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:41.625195 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:41.625239 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:41.627775 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:41.632124 (+ 4349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:41.632131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:41.632145 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:41.632608 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:41.633106 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:41.633109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:41.633112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:41.633310 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:41.634010 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:41.634015 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:41.634372 (+ 357us) client_negotiation.cc:769] Sending connection context
0812 06:55:41.634698 (+ 326us) client_negotiation.cc:240] Negotiation successful
0812 06:55:41.635179 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":287,"threads_started":1}
I20260812 06:55:41.638341 28823 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33385
I20260812 06:55:41.638664 28823 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:41.639420 28823 heartbeater.cc:511] Master 127.26.226.62:33385 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
I20260812 06:55:41.641723 28505 ts_manager.cc:194] Registered new tserver with Master: e864cc0d340848d285b8ca506eaf51b8 (127.26.226.1:36529)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
I20260812 06:55:41.643239 28505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:49105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:55:41.668121 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33385
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:33489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:42.020291 28832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:42.020596 28832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:42.020787 28832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:42.028051 28832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:42.028226 28832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:42.028417 28832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:55:42.038586 28832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:33489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33385
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:42.039944 28832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:42.041373 28832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:42.054816 28840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:42.054860 28838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:42.054898 28844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:42.055656 28832 server_base.cc:1034] running on GCE node
I20260812 06:55:42.056356 28832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:42.057296 28832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:42.058537 28832 hybrid_clock.cc:648] HybridClock initialized: now 1786517742058493 us; error 55 us; skew 500 ppm
I20260812 06:55:42.058858 28832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:42 dist-test-slave-t3q3 krb5kdc[28431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517742, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:42.062017 28832 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:55:42.063423 28832 webserver.cc:460] Webserver started at http://127.26.226.2:35443/ using document root <none> and password file <none>
I20260812 06:55:42.064193 28832 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:42.064352 28832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:42.064672 28832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:42.067589 28832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "438ba6f175e54f0f86d2e18271eea679"
format_stamp: "Formatted at 2026-08-12 06:55:42 on dist-test-slave-t3q3"
server_key: "e807628a45502545d21768b03358d58d"
server_key_iv: "3a76884bc9ed08e0291031d8a4f50acc"
server_key_version: "encryptionkey@0"
I20260812 06:55:42.068468 28832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "438ba6f175e54f0f86d2e18271eea679"
format_stamp: "Formatted at 2026-08-12 06:55:42 on dist-test-slave-t3q3"
server_key: "e807628a45502545d21768b03358d58d"
server_key_iv: "3a76884bc9ed08e0291031d8a4f50acc"
server_key_version: "encryptionkey@0"
I20260812 06:55:42.074054 28832 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260812 06:55:42.077495 28856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.078787 28832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:55:42.078976 28832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "438ba6f175e54f0f86d2e18271eea679"
format_stamp: "Formatted at 2026-08-12 06:55:42 on dist-test-slave-t3q3"
server_key: "e807628a45502545d21768b03358d58d"
server_key_iv: "3a76884bc9ed08e0291031d8a4f50acc"
server_key_version: "encryptionkey@0"
I20260812 06:55:42.079192 28832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:42.094700 28832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:42.098847 28832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:42.099182 28832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:42.100308 28832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:42.101722 28832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:42.101828 28832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.101981 28832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:42.102054 28832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.136745 28832 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33327
I20260812 06:55:42.136770 29043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33327 every 8 connection(s)
I20260812 06:55:42.138253 28832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:42.140877 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 28832
I20260812 06:55:42.141073 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:42.141607 27528 external_mini_cluster.cc:1428] Setting key c22d48a06f7a0f6ff83d429a1972ffa7
Aug 12 06:55:42 dist-test-slave-t3q3 krb5kdc[28431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517742, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:42.159700 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.141748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:55699 (local address 127.26.226.62:33385)
0812 06:55:42.142200 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.142207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.142360 (+ 153us) server_negotiation.cc:407] Connection header received
0812 06:55:42.144689 (+ 2329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.144696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.144778 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.144905 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:42.146398 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.147569 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.149187 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.149912 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.150062 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.153760 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:42.153765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:42.153768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:42.153800 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:42.156620 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:42.157366 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:42.157370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:42.157373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:42.157505 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:42.157964 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:42.157968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:42.157970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:42.158238 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:42.158398 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.159077 (+ 679us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.159304 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":900,"threads_started":1}
I20260812 06:55:42.160774 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.141326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33385 (local address 127.26.226.2:55699)
0812 06:55:42.142151 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.142226 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.144409 (+ 2183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.145102 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.145110 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.145638 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:42.146213 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.146231 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.147730 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.147734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.149033 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.149045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.150125 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.150130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.150338 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.151061 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:42.151085 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:42.153526 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:42.156769 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:42.156778 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:42.156793 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:42.157226 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:42.157631 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:42.157636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:42.157640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:42.157832 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:42.158372 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:42.158380 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:42.158722 (+ 342us) client_negotiation.cc:769] Sending connection context
0812 06:55:42.158988 (+ 266us) client_negotiation.cc:240] Negotiation successful
0812 06:55:42.159522 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":490,"threads_started":1}
I20260812 06:55:42.162258 29044 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33385
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
I20260812 06:55:42.162652 29044 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:42.163458 29044 heartbeater.cc:511] Master 127.26.226.62:33385 requested a full tablet report, sending...
I20260812 06:55:42.165071 28505 ts_manager.cc:194] Registered new tserver with Master: 438ba6f175e54f0f86d2e18271eea679 (127.26.226.2:33327)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
I20260812 06:55:42.166288 28505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:55699
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:55:42.189652 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33385
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:33489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:42.530921 29053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:42.531203 29053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:42.531299 29053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:42.538558 29053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:42.538664 29053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:42.538779 29053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:55:42.549990 29053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:33489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33385
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:42.551247 29053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:42.552621 29053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:42.566174 29065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:42.566134 29061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:42.566215 29060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:42.567133 29053 server_base.cc:1034] running on GCE node
I20260812 06:55:42.567792 29053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:42.568924 29053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:42.570166 29053 hybrid_clock.cc:648] HybridClock initialized: now 1786517742570123 us; error 55 us; skew 500 ppm
I20260812 06:55:42.570501 29053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:42 dist-test-slave-t3q3 krb5kdc[28431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517742, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:42.574008 29053 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:55:42.575582 29053 webserver.cc:460] Webserver started at http://127.26.226.3:44783/ using document root <none> and password file <none>
I20260812 06:55:42.576505 29053 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:42.576647 29053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:42.577014 29053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:42.580001 29053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "2ebb7285bb974750b881509be0aaa601"
format_stamp: "Formatted at 2026-08-12 06:55:42 on dist-test-slave-t3q3"
server_key: "abf0a5eb177422c27f00d2ac10b28260"
server_key_iv: "a98e01c12a98a2ed5a053e422921690b"
server_key_version: "encryptionkey@0"
I20260812 06:55:42.580914 29053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "2ebb7285bb974750b881509be0aaa601"
format_stamp: "Formatted at 2026-08-12 06:55:42 on dist-test-slave-t3q3"
server_key: "abf0a5eb177422c27f00d2ac10b28260"
server_key_iv: "a98e01c12a98a2ed5a053e422921690b"
server_key_version: "encryptionkey@0"
I20260812 06:55:42.586350 29053 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:55:42.590080 29078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.591419 29053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:42.591588 29053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "2ebb7285bb974750b881509be0aaa601"
format_stamp: "Formatted at 2026-08-12 06:55:42 on dist-test-slave-t3q3"
server_key: "abf0a5eb177422c27f00d2ac10b28260"
server_key_iv: "a98e01c12a98a2ed5a053e422921690b"
server_key_version: "encryptionkey@0"
I20260812 06:55:42.591815 29053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:42.616500 29053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:42.620996 29053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:42.621318 29053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:42.622294 29053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:42.623641 29053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:42.623718 29053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.623898 29053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:42.623951 29053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.647195 28823 heartbeater.cc:503] Master 127.26.226.62:33385 was elected leader, sending a full tablet report...
I20260812 06:55:42.658995 29053 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:46645
I20260812 06:55:42.659049 29267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:46645 every 8 connection(s)
I20260812 06:55:42.660401 29053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:42.670866 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 29053
I20260812 06:55:42.671096 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:42.671722 27528 external_mini_cluster.cc:1428] Setting key 81da8fc13d5e08e8552af8863a98a84a
Aug 12 06:55:42 dist-test-slave-t3q3 krb5kdc[28431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517742, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:42.681500 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.662918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:55497 (local address 127.26.226.62:33385)
0812 06:55:42.663414 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.663419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.664294 (+ 875us) server_negotiation.cc:407] Connection header received
0812 06:55:42.666423 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.666428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.666525 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.666668 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:42.668256 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.669082 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.670790 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.671382 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.671494 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.675795 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:42.675800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:42.675803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:42.675836 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:42.678615 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:42.679314 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:42.679318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:42.679320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:42.679443 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:42.679845 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:42.679849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:42.679851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:42.680213 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:42.680349 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.681029 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.681217 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":277,"threads_started":1}
I20260812 06:55:42.682494 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.663381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33385 (local address 127.26.226.3:55497)
0812 06:55:42.664132 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.664179 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.666119 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.666920 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.666928 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.667449 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:42.668023 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.668041 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.669254 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.669267 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.670606 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.670632 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.671625 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.671633 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.671973 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.672921 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:42.672944 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:42.675599 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:42.678747 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:42.678754 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:42.678767 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:42.679199 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:42.679539 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:42.679542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:42.679544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:42.679741 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:42.680331 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:42.680337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:42.680672 (+ 335us) client_negotiation.cc:769] Sending connection context
0812 06:55:42.680918 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 06:55:42.681315 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":278,"threads_started":1}
I20260812 06:55:42.684115 29268 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33385
I20260812 06:55:42.684441 29268 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:42.685168 29268 heartbeater.cc:511] Master 127.26.226.62:33385 requested a full tablet report, sending...
I20260812 06:55:42.686753 28505 ts_manager.cc:194] Registered new tserver with Master: 2ebb7285bb974750b881509be0aaa601 (127.26.226.3:46645)
I20260812 06:55:42.687774 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:55:42.687920 28505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:55497
I20260812 06:55:42.706717 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.694250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51654 (local address 127.26.226.62:33385)
0812 06:55:42.694436 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.694440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.694939 (+ 499us) server_negotiation.cc:407] Connection header received
0812 06:55:42.695098 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.695103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.695176 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.695274 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:42.696919 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.697628 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.698857 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.699251 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.699346 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.701780 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:42.701785 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:42.701788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:42.701818 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:42.704513 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:42.705089 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:42.705093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:42.705095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:42.705221 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:42.705532 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:42.705535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:42.705537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:42.705846 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:42.706024 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.706321 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.706488 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:55:42.713002 28505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:42.722251 28505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:42.767450 29299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.741326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.26.226.3:46645)
0812 06:55:42.753252 (+ 11926us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.753265 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.754007 (+ 742us) server_negotiation.cc:407] Connection header received
0812 06:55:42.754399 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.754414 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.754810 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.755153 (+ 343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:42.757277 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.759359 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.763602 (+ 4243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.765129 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.765297 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.765627 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.766594 (+ 967us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.766860 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11644,"thread_start_us":11347,"threads_started":1}
I20260812 06:55:42.769160 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.740727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:46645 (local address 127.0.0.1:59070)
0812 06:55:42.753816 (+ 13089us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.753847 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.754129 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.755240 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.755253 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.755301 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:42.756634 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.756661 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.759630 (+ 2969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.759639 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.763418 (+ 3779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.763432 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.765492 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.765501 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.766244 (+ 743us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.766312 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 06:55:42.766441 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 06:55:42.767929 (+ 1488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12872,"mutex_wait_us":2,"thread_start_us":12583,"threads_started":1}
I20260812 06:55:42.771862 29300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.742412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:36529 (local address 127.0.0.1:46176)
0812 06:55:42.757681 (+ 15269us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.757707 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.757916 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.759556 (+ 1640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.759564 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.759602 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:42.761210 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.761226 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.764582 (+ 3356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.764602 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.768144 (+ 3542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.768161 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.770755 (+ 2594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.770762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.770940 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.770996 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 06:55:42.771126 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 06:55:42.771280 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15020,"thread_start_us":14592,"threads_started":1}
I20260812 06:55:42.771997 29301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.742454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.26.226.1:36529)
0812 06:55:42.752393 (+ 9939us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.752406 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.758128 (+ 5722us) server_negotiation.cc:407] Connection header received
0812 06:55:42.758272 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.758284 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.758703 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.759010 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:42.761521 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.764314 (+ 2793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.768372 (+ 4058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.770473 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.770652 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.770904 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.771210 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.771408 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9484,"thread_start_us":9070,"threads_started":1}
I20260812 06:55:42.774380 29165 tablet_service.cc:1467] Processing CreateTablet for tablet dd23b9e609f3440486b4dc17eb4a165c (DEFAULT_TABLE table=test-table [id=49f0b19281e94083a2b5aec2a4e0aa6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:42.776259 29165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd23b9e609f3440486b4dc17eb4a165c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:42.776046 28716 tablet_service.cc:1467] Processing CreateTablet for tablet dd23b9e609f3440486b4dc17eb4a165c (DEFAULT_TABLE table=test-table [id=49f0b19281e94083a2b5aec2a4e0aa6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:42.777783 29293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.739925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33327 (local address 127.0.0.1:40672)
0812 06:55:42.741367 (+ 1442us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.741407 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.741631 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.762072 (+ 20441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.762078 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.762131 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:42.763532 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.763549 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.766702 (+ 3153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.766708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.769536 (+ 2828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.769554 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.776489 (+ 6935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.776500 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.776785 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.776824 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 06:55:42.776958 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 06:55:42.777171 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"thread_start_us":874,"threads_started":1}
I20260812 06:55:42.778116 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.740116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.26.226.2:33327)
0812 06:55:42.760845 (+ 20729us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.760857 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.760899 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:55:42.761057 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.761067 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.761400 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.763836 (+ 2436us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:42.764056 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.766481 (+ 2425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.769791 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.776087 (+ 6296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.776278 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.776605 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.777193 (+ 588us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.777392 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20435,"thread_start_us":21509,"threads_started":1}
I20260812 06:55:42.778771 28716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd23b9e609f3440486b4dc17eb4a165c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:42.785161 28935 tablet_service.cc:1467] Processing CreateTablet for tablet dd23b9e609f3440486b4dc17eb4a165c (DEFAULT_TABLE table=test-table [id=49f0b19281e94083a2b5aec2a4e0aa6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:42.786989 28935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd23b9e609f3440486b4dc17eb4a165c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:42.794541 29302 tablet_bootstrap.cc:492] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8: Bootstrap starting.
I20260812 06:55:42.797124 29303 tablet_bootstrap.cc:492] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601: Bootstrap starting.
I20260812 06:55:42.801096 29302 tablet_bootstrap.cc:654] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:42.803218 29303 tablet_bootstrap.cc:654] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:42.803217 29302 log.cc:824] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:42.804908 29304 tablet_bootstrap.cc:492] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679: Bootstrap starting.
I20260812 06:55:42.805470 29303 log.cc:824] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:42.809165 29302 tablet_bootstrap.cc:492] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8: No bootstrap required, opened a new log
I20260812 06:55:42.809535 29302 ts_tablet_manager.cc:1397] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260812 06:55:42.809633 29303 tablet_bootstrap.cc:492] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601: No bootstrap required, opened a new log
I20260812 06:55:42.809979 29303 ts_tablet_manager.cc:1397] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260812 06:55:42.810251 29304 tablet_bootstrap.cc:654] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:42.812168 29304 log.cc:824] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:42.816725 29304 tablet_bootstrap.cc:492] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679: No bootstrap required, opened a new log
I20260812 06:55:42.817119 29304 ts_tablet_manager.cc:1397] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260812 06:55:42.819728 29302 raft_consensus.cc:348] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } }
I20260812 06:55:42.820218 29302 raft_consensus.cc:374] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:42.820374 29302 raft_consensus.cc:729] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e864cc0d340848d285b8ca506eaf51b8, State: Initialized, Role: FOLLOWER
I20260812 06:55:42.821079 29302 consensus_queue.cc:260] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } }
I20260812 06:55:42.823546 29303 raft_consensus.cc:348] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } }
I20260812 06:55:42.824187 29303 raft_consensus.cc:374] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:42.824322 29303 raft_consensus.cc:729] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebb7285bb974750b881509be0aaa601, State: Initialized, Role: FOLLOWER
I20260812 06:55:42.825135 29303 consensus_queue.cc:260] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } }
I20260812 06:55:42.825716 29302 ts_tablet_manager.cc:1428] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260812 06:55:42.827358 29268 heartbeater.cc:503] Master 127.26.226.62:33385 was elected leader, sending a full tablet report...
I20260812 06:55:42.828125 29303 ts_tablet_manager.cc:1428] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601: Time spent starting tablet: real 0.018s user 0.011s sys 0.008s
I20260812 06:55:42.831360 29304 raft_consensus.cc:348] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } }
I20260812 06:55:42.832044 29304 raft_consensus.cc:374] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:42.832300 29304 raft_consensus.cc:729] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438ba6f175e54f0f86d2e18271eea679, State: Initialized, Role: FOLLOWER
I20260812 06:55:42.833040 29304 consensus_queue.cc:260] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } }
I20260812 06:55:42.834295 29044 heartbeater.cc:503] Master 127.26.226.62:33385 was elected leader, sending a full tablet report...
I20260812 06:55:42.835258 29304 ts_tablet_manager.cc:1428] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679: Time spent starting tablet: real 0.018s user 0.009s sys 0.007s
I20260812 06:55:42.861645 29312 raft_consensus.cc:482] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:42.862074 29312 raft_consensus.cc:504] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } }
W20260812 06:55:42.863579 28824 tablet.cc:2369] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:55:42.864511 29312 leader_election.cc:288] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438ba6f175e54f0f86d2e18271eea679 (127.26.226.2:33327), e864cc0d340848d285b8ca506eaf51b8 (127.26.226.1:36529)
I20260812 06:55:42.867827 29311 raft_consensus.cc:482] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:42.868340 29311 raft_consensus.cc:504] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } }
I20260812 06:55:42.873090 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.864623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33327 (local address 127.26.226.3:50249)
0812 06:55:42.864984 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.865011 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.865150 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.865902 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.865909 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.865947 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:42.866185 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.866210 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.867447 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.867454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.870045 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.870061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.872176 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.872190 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.872445 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.872495 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 06:55:42.872591 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 06:55:42.872737 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260812 06:55:42.873811 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.865059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:50249 (local address 127.26.226.2:33327)
0812 06:55:42.865427 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.865435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.865471 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:55:42.865586 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.865595 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.865721 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.865871 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:42.866370 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.867308 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.870253 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.871804 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.871975 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.872056 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.873355 (+ 1299us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.873498 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260812 06:55:42.875387 28966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd23b9e609f3440486b4dc17eb4a165c" candidate_uuid: "2ebb7285bb974750b881509be0aaa601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438ba6f175e54f0f86d2e18271eea679" is_pre_election: true
I20260812 06:55:42.875726 29301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.865230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:38005 (local address 127.26.226.1:36529)
0812 06:55:42.865523 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.865532 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.866267 (+ 735us) server_negotiation.cc:407] Connection header received
0812 06:55:42.866497 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.866501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.866560 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.866659 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:42.867671 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.868861 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.871874 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.873734 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.874104 (+ 370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.874209 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.875125 (+ 916us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.875317 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 06:55:42.876216 28966 raft_consensus.cc:2454] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ebb7285bb974750b881509be0aaa601 in term 0.
I20260812 06:55:42.876907 29316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.865067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:36529 (local address 127.26.226.3:38005)
0812 06:55:42.866139 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.866153 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.866340 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.866676 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.866680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.866714 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:42.867416 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.867430 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.869042 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.869049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.871593 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.871611 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.874530 (+ 2919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.874537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.874750 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.874810 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 06:55:42.876169 (+ 1359us) client_negotiation.cc:240] Negotiation successful
0812 06:55:42.876311 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":597,"threads_started":1}
I20260812 06:55:42.877715 29086 leader_election.cc:302] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebb7285bb974750b881509be0aaa601, 438ba6f175e54f0f86d2e18271eea679; no voters:
I20260812 06:55:42.878036 29311 leader_election.cc:288] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438ba6f175e54f0f86d2e18271eea679 (127.26.226.2:33327), 2ebb7285bb974750b881509be0aaa601 (127.26.226.3:46645)
I20260812 06:55:42.878989 29312 raft_consensus.cc:2781] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:42.879213 29312 raft_consensus.cc:482] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:42.879385 29312 raft_consensus.cc:3037] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:42.879298 28752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd23b9e609f3440486b4dc17eb4a165c" candidate_uuid: "2ebb7285bb974750b881509be0aaa601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e864cc0d340848d285b8ca506eaf51b8" is_pre_election: true
I20260812 06:55:42.880340 28752 raft_consensus.cc:2454] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ebb7285bb974750b881509be0aaa601 in term 0.
I20260812 06:55:42.884128 29312 raft_consensus.cc:504] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } }
I20260812 06:55:42.885418 29312 leader_election.cc:288] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [CANDIDATE]: Term 1 election: Requested vote from peers 438ba6f175e54f0f86d2e18271eea679 (127.26.226.2:33327), e864cc0d340848d285b8ca506eaf51b8 (127.26.226.1:36529)
I20260812 06:55:42.885850 28966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd23b9e609f3440486b4dc17eb4a165c" candidate_uuid: "2ebb7285bb974750b881509be0aaa601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438ba6f175e54f0f86d2e18271eea679"
I20260812 06:55:42.886233 28966 raft_consensus.cc:3037] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:42.886344 28752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd23b9e609f3440486b4dc17eb4a165c" candidate_uuid: "2ebb7285bb974750b881509be0aaa601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e864cc0d340848d285b8ca506eaf51b8"
I20260812 06:55:42.886744 28752 raft_consensus.cc:3037] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:42.887491 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.872125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:35237 (local address 127.26.226.2:33327)
0812 06:55:42.874641 (+ 2516us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.874647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.875851 (+ 1204us) server_negotiation.cc:407] Connection header received
0812 06:55:42.876173 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.876190 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.876265 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.876326 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:42.877702 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.879105 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.883555 (+ 4450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.885376 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.885578 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.885649 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.886669 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.886841 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2416,"thread_start_us":5439,"threads_started":1}
I20260812 06:55:42.887728 29318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.870909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33327 (local address 127.26.226.1:35237)
0812 06:55:42.875745 (+ 4836us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.875767 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.875976 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.876579 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.876584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.876613 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:42.877450 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.877477 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.879292 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.879312 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.883333 (+ 4021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.883351 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.886003 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.886013 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.886324 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.886361 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 06:55:42.886529 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 06:55:42.886713 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4622,"thread_start_us":6811,"threads_started":1}
W20260812 06:55:42.890595 29045 tablet.cc:2369] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:55:42.891712 28752 raft_consensus.cc:2454] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebb7285bb974750b881509be0aaa601 in term 1.
I20260812 06:55:42.892462 29086 leader_election.cc:302] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebb7285bb974750b881509be0aaa601, e864cc0d340848d285b8ca506eaf51b8; no voters:
I20260812 06:55:42.892562 28963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd23b9e609f3440486b4dc17eb4a165c" candidate_uuid: "e864cc0d340848d285b8ca506eaf51b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438ba6f175e54f0f86d2e18271eea679" is_pre_election: true
I20260812 06:55:42.893329 29312 raft_consensus.cc:2781] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:42.894057 28966 raft_consensus.cc:2454] T dd23b9e609f3440486b4dc17eb4a165c P 438ba6f175e54f0f86d2e18271eea679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebb7285bb974750b881509be0aaa601 in term 1.
I20260812 06:55:42.894392 29299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.871386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:39187 (local address 127.26.226.3:46645)
0812 06:55:42.871828 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.871834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.879191 (+ 7357us) server_negotiation.cc:407] Connection header received
0812 06:55:42.879731 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.879739 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.879822 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.879922 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:42.881833 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.883176 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.890472 (+ 7296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.892598 (+ 2126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.892804 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.892884 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.893813 (+ 929us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.894013 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260812 06:55:42.894537 29319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.871402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:46645 (local address 127.26.226.1:39187)
0812 06:55:42.879030 (+ 7628us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.879052 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.879429 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.880326 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.880331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.880361 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:42.881518 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.881534 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.883441 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.883446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.890174 (+ 6728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.890192 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.893270 (+ 3078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.893278 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.893543 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.893576 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:55:42.893734 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 06:55:42.893904 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7473,"thread_start_us":7146,"threads_started":1}
I20260812 06:55:42.895689 29312 raft_consensus.cc:686] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [term 1 LEADER]: Becoming Leader. State: Replica: 2ebb7285bb974750b881509be0aaa601, State: Running, Role: LEADER
I20260812 06:55:42.895952 29195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd23b9e609f3440486b4dc17eb4a165c" candidate_uuid: "e864cc0d340848d285b8ca506eaf51b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebb7285bb974750b881509be0aaa601" is_pre_election: true
I20260812 06:55:42.896509 29312 consensus_queue.cc:237] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } }
I20260812 06:55:42.898260 28641 leader_election.cc:302] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e864cc0d340848d285b8ca506eaf51b8; no voters: 2ebb7285bb974750b881509be0aaa601, 438ba6f175e54f0f86d2e18271eea679
I20260812 06:55:42.898806 29311 raft_consensus.cc:2726] T dd23b9e609f3440486b4dc17eb4a165c P e864cc0d340848d285b8ca506eaf51b8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 06:55:42.902921 28504 catalog_manager.cc:5495] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ebb7285bb974750b881509be0aaa601 (127.26.226.3). New cstate: current_term: 1 leader_uuid: "2ebb7285bb974750b881509be0aaa601" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438ba6f175e54f0f86d2e18271eea679" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e864cc0d340848d285b8ca506eaf51b8" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebb7285bb974750b881509be0aaa601" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46645 } health_report { overall_health: HEALTHY } } }
W20260812 06:55:42.912266 29269 tablet.cc:2369] T dd23b9e609f3440486b4dc17eb4a165c P 2ebb7285bb974750b881509be0aaa601: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 06:55:42 dist-test-slave-t3q3 krb5kdc[28431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:55:42.940727 29301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.918064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46178 (local address 127.26.226.1:36529)
0812 06:55:42.918273 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.918278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.919648 (+ 1370us) server_negotiation.cc:407] Connection header received
0812 06:55:42.919912 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.919917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.920004 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.920140 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:42.922949 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.924013 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.925648 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.926211 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.926357 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.932625 (+ 6268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:42.932640 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:42.932657 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:42.932717 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:42.937230 (+ 4513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:42.938069 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:42.938077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:42.938079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:42.938350 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:42.938758 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:42.938761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:42.938763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:42.939315 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:42.939615 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:42.939897 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 06:55:42.940131 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 06:55:42 dist-test-slave-t3q3 krb5kdc[28431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:55:42 dist-test-slave-t3q3 krb5kdc[28431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:55:43.003721 29301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.983700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46194 (local address 127.26.226.1:36529)
0812 06:55:42.983940 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.983949 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.984365 (+ 416us) server_negotiation.cc:407] Connection header received
0812 06:55:42.984592 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.984597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.984671 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.984791 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:42.986797 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.987991 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.990107 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.990871 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.991061 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.996759 (+ 5698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:42.996765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:42.996768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:42.996804 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:43.000506 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:43.001195 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:43.001203 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:43.001209 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:43.001408 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:43.001913 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:43.001918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:43.001921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:43.002289 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:43.002740 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:43.002882 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 06:55:43.003110 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 06:55:43 dist-test-slave-t3q3 krb5kdc[28431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:55:43 dist-test-slave-t3q3 krb5kdc[28431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:55:43.062546 29301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:43.042989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.26.226.1:36529)
0812 06:55:43.043261 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:43.043271 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:43.043626 (+ 355us) server_negotiation.cc:407] Connection header received
0812 06:55:43.043857 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:43.043861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:43.043927 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:43.044038 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:43.045458 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.046461 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.048204 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.048802 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.048953 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:43.054718 (+ 5765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:43.054723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:43.054727 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:43.054765 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:43.058415 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:43.059333 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:43.059338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:43.059340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:43.059464 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:43.059957 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:43.059962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:43.059964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:43.060584 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:43.061058 (+ 474us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:43.061469 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 06:55:43.061850 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 06:55:43.064746 28682 tablet_service.cc:2908] Rejecting scan request for tablet dd23b9e609f3440486b4dc17eb4a165c: Uninitialized: clean time has not yet been initialized
I20260812 06:55:43.068435 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 28607
I20260812 06:55:43.084347 29301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:43.069313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46214 (local address 127.26.226.1:36529)
0812 06:55:43.069537 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:43.069544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:43.069774 (+ 230us) server_negotiation.cc:407] Connection header received
0812 06:55:43.069895 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:43.069899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:43.069970 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:43.070050 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:43.071160 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.072081 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.073607 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.074394 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.074523 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:43.077520 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:43.077525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:43.077528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:43.077560 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:43.081228 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:43.081997 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:43.082001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:43.082004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:43.082197 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:43.082611 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:43.082615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:43.082617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:43.083061 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:43.083454 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:43.083607 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 06:55:43.083835 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:55:43.085234 28813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:43.181671 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 28607
I20260812 06:55:43.206897 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 28832
Aug 12 06:55:43 dist-test-slave-t3q3 krb5kdc[28431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:55:43.221622 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:43.207744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.26.226.2:33327)
0812 06:55:43.207918 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:43.207923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:43.207943 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:43.208143 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:43.208147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:43.208228 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:43.208319 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:43.209714 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.210496 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.211746 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.212269 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.212370 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:43.216336 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:43.216349 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:43.216365 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:43.216396 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:43.218999 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:43.219621 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:43.219628 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:43.219631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:43.219749 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:43.220362 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:43.220366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:43.220368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:43.220720 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:43.220964 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:43.221212 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 06:55:43.221399 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:55:43.222309 29036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:43.312045 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 28832
I20260812 06:55:43.339154 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29053
Aug 12 06:55:43 dist-test-slave-t3q3 krb5kdc[28431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:55:43.354125 29299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:43.339928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.26.226.3:46645)
0812 06:55:43.340151 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:43.340156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:43.340175 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:55:43.340291 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:43.340295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:43.340361 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:43.340447 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:43.341664 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.342392 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.343631 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.344010 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.344276 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:43.348240 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:43.348255 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:43.348272 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:43.348309 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:43.351258 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:43.351866 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:43.351873 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:43.351876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:43.352164 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:43.352493 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:43.352497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:43.352499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:43.352987 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:43.353304 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:43.353479 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 06:55:43.353639 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:55:43.354840 29260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:43.445281 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 29053
I20260812 06:55:43.469981 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 28448
I20260812 06:55:43.471235 28558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:43.582072 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 28448
2026-08-12T06:55:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4641 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:43 dist-test-slave-t3q3 krb5kdc[29358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:43 dist-test-slave-t3q3 krb5kdc[29358](info): set up 2 sockets
Aug 12 06:55:43 dist-test-slave-t3q3 krb5kdc[29358](info): commencing operation
krb5kdc: starting...
W20260812 06:55:44.923880 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:55:45 dist-test-slave-t3q3 krb5kdc[29358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:55:45.163275 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:41415
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:41415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:45.496359 29380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:45.496726 29380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:45.496893 29380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:45.503993 29380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:45.504180 29380 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 06:55:45.504277 29380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:45.504396 29380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:45.504492 29380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:45.504585 29380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:45.514624 29380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:41415
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:41415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:45.515981 29380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:45.517400 29380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:45.529577 29386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:45.529506 29394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:45.529639 29391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:45.530869 29380 server_base.cc:1034] running on GCE node
I20260812 06:55:45.531734 29380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:45.533294 29380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:45.534531 29380 hybrid_clock.cc:648] HybridClock initialized: now 1786517745534433 us; error 63 us; skew 500 ppm
I20260812 06:55:45.534905 29380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:45 dist-test-slave-t3q3 krb5kdc[29358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517745, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:45.538209 29380 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:55:45.539845 29380 webserver.cc:460] Webserver started at http://127.26.226.62:45447/ using document root <none> and password file <none>
I20260812 06:55:45.540635 29380 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:45.540729 29380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:45.541028 29380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:45.544256 29380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "23a7eb92554e4a2f9478b4445016c257"
format_stamp: "Formatted at 2026-08-12 06:55:45 on dist-test-slave-t3q3"
server_key: "8e0481826b1489dc8ef2a39f30759d22"
server_key_iv: "752264d960faffd9850ca1c8c076f3db"
server_key_version: "encryptionkey@0"
I20260812 06:55:45.545049 29380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "23a7eb92554e4a2f9478b4445016c257"
format_stamp: "Formatted at 2026-08-12 06:55:45 on dist-test-slave-t3q3"
server_key: "8e0481826b1489dc8ef2a39f30759d22"
server_key_iv: "752264d960faffd9850ca1c8c076f3db"
server_key_version: "encryptionkey@0"
I20260812 06:55:45.550556 29380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:55:45.555534 29413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:45.557053 29380 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260812 06:55:45.557278 29380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "23a7eb92554e4a2f9478b4445016c257"
format_stamp: "Formatted at 2026-08-12 06:55:45 on dist-test-slave-t3q3"
server_key: "8e0481826b1489dc8ef2a39f30759d22"
server_key_iv: "752264d960faffd9850ca1c8c076f3db"
server_key_version: "encryptionkey@0"
I20260812 06:55:45.557538 29380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:45.579064 29380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:45.583752 29380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:45.584084 29380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:45.606312 29380 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:41415
I20260812 06:55:45.606336 29512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:41415 every 8 connection(s)
I20260812 06:55:45.607811 29380 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:45.611992 29513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:45.615562 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 29380
I20260812 06:55:45.615818 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:55:45.616731 27528 external_mini_cluster.cc:1428] Setting key a42eaba8413ea3f6a4d889b51a5fb708
I20260812 06:55:45.625996 29513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257: Bootstrap starting.
I20260812 06:55:45.630026 29513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:45.631435 29513 log.cc:824] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:55:45 dist-test-slave-t3q3 krb5kdc[29358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:45.634770 29513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257: No bootstrap required, opened a new log
I20260812 06:55:45.641739 29517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:45.619097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34234 (local address 127.26.226.62:41415)
0812 06:55:45.619917 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:45.619928 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:45.619969 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:55:45.621749 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:45.621773 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:45.622185 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:45.622555 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:45.624382 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.625736 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.627508 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.628380 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.628541 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:45.633964 (+ 5423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:45.633972 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:45.633986 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:45.634022 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:45.637409 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:45.638241 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:45.638247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:45.638253 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:45.638423 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:45.638843 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:45.638847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:45.638849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:45.639401 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:45.639618 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:45.640183 (+ 565us) server_negotiation.cc:299] Negotiation successful
0812 06:55:45.640637 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":388,"threads_started":1}
I20260812 06:55:45.644200 29513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23a7eb92554e4a2f9478b4445016c257" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41415 } }
I20260812 06:55:45.644609 29513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:45.644745 29513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23a7eb92554e4a2f9478b4445016c257, State: Initialized, Role: FOLLOWER
I20260812 06:55:45.645370 29513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23a7eb92554e4a2f9478b4445016c257" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41415 } }
I20260812 06:55:45.645646 29513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:45.645790 29513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:45.645977 29513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:45.648222 29513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23a7eb92554e4a2f9478b4445016c257" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41415 } }
I20260812 06:55:45.648797 29513 leader_election.cc:302] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23a7eb92554e4a2f9478b4445016c257; no voters:
I20260812 06:55:45.649500 29513 leader_election.cc:288] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:45.649621 29519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:45.650324 29519 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [term 1 LEADER]: Becoming Leader. State: Replica: 23a7eb92554e4a2f9478b4445016c257, State: Running, Role: LEADER
I20260812 06:55:45.650955 29519 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23a7eb92554e4a2f9478b4445016c257" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41415 } }
I20260812 06:55:45.651857 29513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:45.654253 29520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23a7eb92554e4a2f9478b4445016c257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23a7eb92554e4a2f9478b4445016c257" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41415 } } }
I20260812 06:55:45.655036 29520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:45.654606 29525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23a7eb92554e4a2f9478b4445016c257. Latest consensus state: current_term: 1 leader_uuid: "23a7eb92554e4a2f9478b4445016c257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23a7eb92554e4a2f9478b4445016c257" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41415 } } }
I20260812 06:55:45.655820 29525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:45.661430 29534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:45.667163 29534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:45.677300 29534 catalog_manager.cc:1347] Generated new cluster ID: 9966ee9e70fd47f397ac5e30428c7c0c
I20260812 06:55:45.677433 29534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:45.688086 29534 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:45.689440 29534 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:45.699095 29534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257: Generated new TSK 0
I20260812 06:55:45.700214 29534 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:55:45.757743 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:41415
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:46.103127 29559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:46.103456 29559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:46.103639 29559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:46.110926 29559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:46.111099 29559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:46.111246 29559 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:55:46.111399 29559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:55:46.121322 29559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.226.62:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:46.122666 29559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:46.124001 29559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:46.137720 29568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:46.137674 29572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:46.137763 29569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:46.139241 29559 server_base.cc:1034] running on GCE node
I20260812 06:55:46.139971 29559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:46.141132 29559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:46.142364 29559 hybrid_clock.cc:648] HybridClock initialized: now 1786517746142300 us; error 37 us; skew 500 ppm
I20260812 06:55:46.142690 29559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:46 dist-test-slave-t3q3 krb5kdc[29358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517746, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:46.150661 29559 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:55:46.152436 29559 webserver.cc:460] Webserver started at http://127.26.226.1:44219/ using document root <none> and password file <none>
I20260812 06:55:46.153342 29559 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:46.153466 29559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:46.153834 29559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:46.157088 29559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "dd6677cf9bdf432088c15d264f78b94d"
format_stamp: "Formatted at 2026-08-12 06:55:46 on dist-test-slave-t3q3"
server_key: "08d347c926353ed9b7200f996767f18c"
server_key_iv: "8cb88a8198cb840cad8557a647a0acd2"
server_key_version: "encryptionkey@0"
I20260812 06:55:46.157998 29559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "dd6677cf9bdf432088c15d264f78b94d"
format_stamp: "Formatted at 2026-08-12 06:55:46 on dist-test-slave-t3q3"
server_key: "08d347c926353ed9b7200f996767f18c"
server_key_iv: "8cb88a8198cb840cad8557a647a0acd2"
server_key_version: "encryptionkey@0"
I20260812 06:55:46.163739 29559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:55:46.167367 29587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:46.168875 29559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:46.169075 29559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "dd6677cf9bdf432088c15d264f78b94d"
format_stamp: "Formatted at 2026-08-12 06:55:46 on dist-test-slave-t3q3"
server_key: "08d347c926353ed9b7200f996767f18c"
server_key_iv: "8cb88a8198cb840cad8557a647a0acd2"
server_key_version: "encryptionkey@0"
I20260812 06:55:46.169327 29559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:46.190167 29559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:46.194799 29559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:46.195114 29559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:46.198257 29559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:46.198375 29559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:46.198551 29559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:46.198632 29559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:55:46 dist-test-slave-t3q3 krb5kdc[29358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517746, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:46.226456 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.199201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:58841 (local address 127.26.226.62:41415)
0812 06:55:46.199811 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:46.199819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:46.201309 (+ 1490us) server_negotiation.cc:407] Connection header received
0812 06:55:46.204740 (+ 3431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:46.204748 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:46.204861 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:46.205002 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:46.207589 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.208698 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.212343 (+ 3645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.212796 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.212907 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.218398 (+ 5491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:46.218407 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:46.218414 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:46.218472 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:46.222011 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:46.223153 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:46.223161 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:46.223166 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:46.223327 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:46.224030 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:46.224034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:46.224036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:46.224522 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:46.224684 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:46.225789 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0812 06:55:46.226048 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":376,"threads_started":1}
I20260812 06:55:46.228037 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.200115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:41415 (local address 127.26.226.1:58841)
0812 06:55:46.201084 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:46.201146 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:46.204248 (+ 3102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:46.205334 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:46.205353 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:46.206298 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:46.207313 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.207351 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.208916 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.208924 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.212126 (+ 3202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.212151 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.213097 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.213109 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.213515 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.214518 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:46.214562 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:46.218122 (+ 3560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:46.222180 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:46.222194 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:46.222221 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:46.222930 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:46.223471 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:46.223476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:46.223480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:46.223845 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:46.224656 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:46.224664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:46.225314 (+ 650us) client_negotiation.cc:769] Sending connection context
0812 06:55:46.225673 (+ 359us) client_negotiation.cc:240] Negotiation successful
0812 06:55:46.226258 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":350,"threads_started":1}
I20260812 06:55:46.250756 29559 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:41185
I20260812 06:55:46.250809 29785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:41185 every 8 connection(s)
I20260812 06:55:46.252682 29559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:46.255167 29786 heartbeater.cc:347] Connected to a master server at 127.26.226.62:41415
I20260812 06:55:46.255504 29786 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:46.256354 29786 heartbeater.cc:511] Master 127.26.226.62:41415 requested a full tablet report, sending...
I20260812 06:55:46.257369 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.239606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:41415 (local address 127.26.226.1:53095)
0812 06:55:46.240458 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:46.240484 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:46.240776 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:46.241380 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:46.241391 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:46.241901 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:46.242227 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.242238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.243883 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.243889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.245964 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.245987 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.247242 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.247250 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.247480 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.248481 (+ 1001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:46.248511 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:46.249392 (+ 881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:46.254219 (+ 4827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:46.254227 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:46.254232 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:46.254792 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:46.255311 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:46.255316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:46.255319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:46.255498 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:46.256099 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:46.256104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:46.256342 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 06:55:46.256512 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 06:55:46.256890 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":848,"threads_started":1}
I20260812 06:55:46.258158 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.240232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:53095 (local address 127.26.226.62:41415)
0812 06:55:46.240458 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:46.240468 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:46.240616 (+ 148us) server_negotiation.cc:407] Connection header received
0812 06:55:46.241033 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:46.241040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:46.241157 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:46.241295 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:46.242400 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.243705 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.246158 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.246990 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.247170 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.249648 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:46.249659 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:46.249667 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:46.249723 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:46.253728 (+ 4005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:46.254965 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:46.254970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:46.254973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:46.255167 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:46.255650 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:46.255656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:46.255659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:46.255945 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:46.256157 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:46.257707 (+ 1550us) server_negotiation.cc:299] Negotiation successful
0812 06:55:46.257904 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:55:46.258868 29448 ts_manager.cc:194] Registered new tserver with Master: dd6677cf9bdf432088c15d264f78b94d (127.26.226.1:41185)
I20260812 06:55:46.259976 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 29559
I20260812 06:55:46.260241 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:46.260912 29448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:58841
I20260812 06:55:46.261149 27528 external_mini_cluster.cc:1428] Setting key 22f96de30c1f14f39d0a25b34d4ddba6
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:55:46.309478 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:41415
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:46.658808 29790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:46.659138 29790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:46.659312 29790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:46.666846 29790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:46.667016 29790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:46.667178 29790 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:55:46.667323 29790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:55:46.677692 29790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.226.62:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:46.679077 29790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:46.680564 29790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:46.694075 29800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:46.694458 29798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:46.694494 29803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:46.695355 29790 server_base.cc:1034] running on GCE node
I20260812 06:55:46.696013 29790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:46.697042 29790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:46.698305 29790 hybrid_clock.cc:648] HybridClock initialized: now 1786517746698237 us; error 43 us; skew 500 ppm
I20260812 06:55:46.698616 29790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:46 dist-test-slave-t3q3 krb5kdc[29358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517746, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:46.702215 29790 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:55:46.703668 29790 webserver.cc:460] Webserver started at http://127.26.226.2:44781/ using document root <none> and password file <none>
I20260812 06:55:46.704450 29790 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:46.704586 29790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:46.704921 29790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:46.707839 29790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "1b965b78dfba4ebeb071920bfee89363"
format_stamp: "Formatted at 2026-08-12 06:55:46 on dist-test-slave-t3q3"
server_key: "f6aaf75ed40cd5cf0f77c2ce0ea1818e"
server_key_iv: "63261f3b611c04eb903b98c38a4a6e1e"
server_key_version: "encryptionkey@0"
I20260812 06:55:46.708729 29790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "1b965b78dfba4ebeb071920bfee89363"
format_stamp: "Formatted at 2026-08-12 06:55:46 on dist-test-slave-t3q3"
server_key: "f6aaf75ed40cd5cf0f77c2ce0ea1818e"
server_key_iv: "63261f3b611c04eb903b98c38a4a6e1e"
server_key_version: "encryptionkey@0"
I20260812 06:55:46.713869 29790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260812 06:55:46.717320 29812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:46.718592 29790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:55:46.718775 29790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "1b965b78dfba4ebeb071920bfee89363"
format_stamp: "Formatted at 2026-08-12 06:55:46 on dist-test-slave-t3q3"
server_key: "f6aaf75ed40cd5cf0f77c2ce0ea1818e"
server_key_iv: "63261f3b611c04eb903b98c38a4a6e1e"
server_key_version: "encryptionkey@0"
I20260812 06:55:46.719007 29790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:46.761467 29790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:46.765950 29790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:46.766479 29790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:46.769388 29790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:46.769477 29790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:46.769676 29790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:46.769740 29790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:55:46 dist-test-slave-t3q3 krb5kdc[29358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517746, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:46.796464 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.770181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:46151 (local address 127.26.226.62:41415)
0812 06:55:46.771820 (+ 1639us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:46.771827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:46.771983 (+ 156us) server_negotiation.cc:407] Connection header received
0812 06:55:46.774634 (+ 2651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:46.774644 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:46.774765 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:46.774907 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:46.776751 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.778133 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.780349 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.781166 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.781346 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.787572 (+ 6226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:46.787580 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:46.787587 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:46.787640 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:46.791735 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:46.793071 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:46.793080 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:46.793087 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:46.793269 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:46.793965 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:46.793973 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:46.793979 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:46.794418 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:46.794663 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:46.795544 (+ 881us) server_negotiation.cc:299] Negotiation successful
0812 06:55:46.795892 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1507,"thread_start_us":1386,"threads_started":1}
I20260812 06:55:46.797590 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.770805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:41415 (local address 127.26.226.2:46151)
0812 06:55:46.771827 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:46.771866 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:46.774259 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:46.775230 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:46.775243 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:46.775903 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:46.776548 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.776566 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.778299 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.778306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.780044 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.780104 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.781758 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.781766 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.782177 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.783182 (+ 1005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:46.783225 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:46.787235 (+ 4010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:46.791941 (+ 4706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:46.791957 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:46.791983 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:46.792884 (+ 901us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:46.793407 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:46.793414 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:46.793421 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:46.793784 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:46.794643 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:46.794654 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:46.795082 (+ 428us) client_negotiation.cc:769] Sending connection context
0812 06:55:46.795337 (+ 255us) client_negotiation.cc:240] Negotiation successful
0812 06:55:46.795762 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":422,"threads_started":1}
I20260812 06:55:46.818246 29790 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33957
I20260812 06:55:46.818272 30007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33957 every 8 connection(s)
I20260812 06:55:46.819681 29790 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:46.822214 30011 heartbeater.cc:347] Connected to a master server at 127.26.226.62:41415
I20260812 06:55:46.822362 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 29790
I20260812 06:55:46.822489 30011 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:46.822569 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:46.823100 27528 external_mini_cluster.cc:1428] Setting key dc80dd74fe26ffe5255de8e4248baba4
I20260812 06:55:46.823446 30011 heartbeater.cc:511] Master 127.26.226.62:41415 requested a full tablet report, sending...
I20260812 06:55:46.825763 29448 ts_manager.cc:194] Registered new tserver with Master: 1b965b78dfba4ebeb071920bfee89363 (127.26.226.2:33957)
I20260812 06:55:46.826114 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.809043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:47697 (local address 127.26.226.62:41415)
0812 06:55:46.809246 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:46.809254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:46.809933 (+ 679us) server_negotiation.cc:407] Connection header received
0812 06:55:46.810306 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:46.810315 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:46.810419 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:46.810568 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:46.812209 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.813574 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.816254 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.816703 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.816857 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.819120 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:46.819126 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:46.819129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:46.819162 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:46.822747 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:46.823710 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:46.823715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:46.823719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:46.823887 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:46.824371 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:46.824376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:46.824378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:46.824747 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:46.824935 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:46.825445 (+ 510us) server_negotiation.cc:299] Negotiation successful
0812 06:55:46.825686 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 06:55:46.826128 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.809056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:41415 (local address 127.26.226.2:47697)
0812 06:55:46.809814 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:46.809834 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:46.810015 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:46.810626 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:46.810632 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:46.811369 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:46.811996 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.812017 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.813809 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.813821 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.816101 (+ 2280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.816120 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.816926 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.816935 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.817157 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.817945 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:46.817966 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:46.818910 (+ 944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:46.822898 (+ 3988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:46.822906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:46.822910 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:46.823578 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:46.824036 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:46.824041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:46.824044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:46.824252 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:46.824913 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:46.824918 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:46.825128 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 06:55:46.825297 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 06:55:46.825665 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":409,"threads_started":1}
I20260812 06:55:46.826714 29448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:46151
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:55:46.873508 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:41415
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:47.215869 30017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:47.216276 30017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:47.216461 30017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:47.224148 30017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:47.224313 30017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:47.224458 30017 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:55:47.224602 30017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:55:47.234696 30017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.226.62:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:47.236114 30017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:47.237470 30017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:47.255271 30027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:47.255275 30028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:47.255275 30030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:47.257185 30017 server_base.cc:1034] running on GCE node
I20260812 06:55:47.257882 30017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:47.258927 30017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:47.260178 30017 hybrid_clock.cc:648] HybridClock initialized: now 1786517747260099 us; error 57 us; skew 500 ppm
I20260812 06:55:47.260512 30017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:47 dist-test-slave-t3q3 krb5kdc[29358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517747, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:47.264302 29786 heartbeater.cc:503] Master 127.26.226.62:41415 was elected leader, sending a full tablet report...
I20260812 06:55:47.264643 30017 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:55:47.266599 30017 webserver.cc:460] Webserver started at http://127.26.226.3:39429/ using document root <none> and password file <none>
I20260812 06:55:47.267352 30017 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:47.267493 30017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:47.267838 30017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:47.270859 30017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "ef7c6113d53f41cca4a808b7e620c9f9"
format_stamp: "Formatted at 2026-08-12 06:55:47 on dist-test-slave-t3q3"
server_key: "db76fbdd3d9e560d4bac08dba22b9c47"
server_key_iv: "9ff311bf965dc7b469ce3caa5e933e7c"
server_key_version: "encryptionkey@0"
I20260812 06:55:47.271685 30017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "ef7c6113d53f41cca4a808b7e620c9f9"
format_stamp: "Formatted at 2026-08-12 06:55:47 on dist-test-slave-t3q3"
server_key: "db76fbdd3d9e560d4bac08dba22b9c47"
server_key_iv: "9ff311bf965dc7b469ce3caa5e933e7c"
server_key_version: "encryptionkey@0"
I20260812 06:55:47.277477 30017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260812 06:55:47.281332 30043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:47.282778 30017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:55:47.282985 30017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "ef7c6113d53f41cca4a808b7e620c9f9"
format_stamp: "Formatted at 2026-08-12 06:55:47 on dist-test-slave-t3q3"
server_key: "db76fbdd3d9e560d4bac08dba22b9c47"
server_key_iv: "9ff311bf965dc7b469ce3caa5e933e7c"
server_key_version: "encryptionkey@0"
I20260812 06:55:47.283255 30017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:47.304194 30017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:47.309058 30017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:47.309387 30017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:47.312433 30017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:47.312556 30017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:47.312736 30017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:47.312824 30017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:55:47 dist-test-slave-t3q3 krb5kdc[29358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517747, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:47.340428 30056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.313889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:41415 (local address 127.26.226.3:51257)
0812 06:55:47.314845 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.314909 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.317861 (+ 2952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.318745 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.318753 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.319576 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:47.320521 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.320551 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.322130 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.322134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.324374 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.324388 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.325433 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.325446 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.325837 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.326698 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:47.326737 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:47.330045 (+ 3308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:47.334950 (+ 4905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:47.334963 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:47.334982 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:47.335650 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:47.336210 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:47.336218 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:47.336224 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:47.336610 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:47.337516 (+ 906us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:47.337531 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:47.338074 (+ 543us) client_negotiation.cc:769] Sending connection context
0812 06:55:47.338507 (+ 433us) client_negotiation.cc:240] Negotiation successful
0812 06:55:47.338946 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":387,"threads_started":1}
I20260812 06:55:47.340935 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.313298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:51257 (local address 127.26.226.62:41415)
0812 06:55:47.313511 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.313518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.315071 (+ 1553us) server_negotiation.cc:407] Connection header received
0812 06:55:47.318239 (+ 3168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.318247 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.318343 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.318489 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:47.320790 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.321925 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.324540 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.325129 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.325277 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.330362 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:47.330372 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:47.330380 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:47.330432 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:47.334732 (+ 4300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:47.335795 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:47.335804 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:47.335810 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:47.335998 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:47.336813 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:47.336821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:47.336827 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:47.337306 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:47.337520 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.340300 (+ 2780us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.340567 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 06:55:47.364569 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.348408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:34573 (local address 127.26.226.62:41415)
0812 06:55:47.348580 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.348588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.349335 (+ 747us) server_negotiation.cc:407] Connection header received
0812 06:55:47.349458 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.349466 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.349584 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.349721 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:47.350635 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.352134 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.354364 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.354955 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.355112 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.357689 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:47.357697 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:47.357703 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:47.357746 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:47.361281 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:47.362404 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:47.362408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:47.362412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:47.362649 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:47.363101 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:47.363104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:47.363106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:47.363388 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:47.363509 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.364122 (+ 613us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.364297 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:55:47.365218 30171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.348513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:41415 (local address 127.26.226.3:34573)
0812 06:55:47.349217 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.349234 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.349353 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.349736 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.349739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.350194 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:47.350487 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.350496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.352308 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.352317 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.354186 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.354206 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.355203 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.355212 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.355483 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.356345 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:47.356367 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:47.357452 (+ 1085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:47.361450 (+ 3998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:47.361460 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:47.361467 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:47.362266 (+ 799us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:47.362767 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:47.362775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:47.362781 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:47.362982 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:47.363518 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:47.363526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:47.363792 (+ 266us) client_negotiation.cc:769] Sending connection context
0812 06:55:47.364007 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 06:55:47.364549 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":367,"threads_started":1}
I20260812 06:55:47.375372 30017 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:39901
I20260812 06:55:47.375417 30228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:39901 every 8 connection(s)
I20260812 06:55:47.376688 30017 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:47.378221 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 30017
I20260812 06:55:47.378432 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:47.379002 27528 external_mini_cluster.cc:1428] Setting key f15cd1f717b47c27618622f18801b66d
I20260812 06:55:47.379021 30229 heartbeater.cc:347] Connected to a master server at 127.26.226.62:41415
I20260812 06:55:47.379261 30229 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:47.380134 30229 heartbeater.cc:511] Master 127.26.226.62:41415 requested a full tablet report, sending...
I20260812 06:55:47.381568 29448 ts_manager.cc:194] Registered new tserver with Master: ef7c6113d53f41cca4a808b7e620c9f9 (127.26.226.3:39901)
I20260812 06:55:47.382632 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:55:47.382642 29448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:51257
Aug 12 06:55:47 dist-test-slave-t3q3 krb5kdc[29358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517747, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:55:47 dist-test-slave-t3q3 krb5kdc[29358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517747, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:47.436973 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.422634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34248 (local address 127.26.226.62:41415)
0812 06:55:47.422895 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.422902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.423087 (+ 185us) server_negotiation.cc:407] Connection header received
0812 06:55:47.423331 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.423336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.423403 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.423514 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:47.425033 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.425777 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.427070 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.427488 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.427632 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.431710 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:47.431715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:47.431719 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:47.431751 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:47.434575 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:47.435147 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:47.435151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:47.435153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:47.435280 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:47.435602 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:47.435605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:47.435607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:47.435891 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:47.436162 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.436401 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.436574 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 06:55:47.441165 29448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:47.448510 29448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:47.486047 30256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.461870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.26.226.2:33957)
0812 06:55:47.476327 (+ 14457us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.476340 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.476378 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:55:47.476522 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.476534 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.476875 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.477164 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:47.478092 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.479941 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.483423 (+ 3482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.484604 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.484768 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.485011 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.485561 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.485742 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14244,"thread_start_us":14052,"threads_started":1}
I20260812 06:55:47.486168 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.461614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33957 (local address 127.0.0.1:59982)
0812 06:55:47.473762 (+ 12148us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.473821 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.474015 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.477207 (+ 3192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.477218 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.477276 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:47.477826 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.477843 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.480187 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.480192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.483221 (+ 3029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.483236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.484874 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.484887 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.485150 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.485324 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 06:55:47.485475 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 06:55:47.485687 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11868,"mutex_wait_us":72,"thread_start_us":19189,"threads_started":1}
I20260812 06:55:47.488729 30255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.461885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41185 (local address 127.0.0.1:36454)
0812 06:55:47.479670 (+ 17785us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.479701 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.480399 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.481460 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.481465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.481490 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:47.482073 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.482090 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.484141 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.484146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.486262 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.486276 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.487906 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.487913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.488095 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.488127 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:55:47.488257 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 06:55:47.488383 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17590,"thread_start_us":17387,"threads_started":1}
I20260812 06:55:47.488862 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.462508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36454 (local address 127.26.226.1:41185)
0812 06:55:47.475742 (+ 13234us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.475757 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.479837 (+ 4080us) server_negotiation.cc:407] Connection header received
0812 06:55:47.480674 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.480687 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.480953 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.481155 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:47.482271 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.483939 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.486455 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.487658 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.487839 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.488046 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.488492 (+ 446us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.488618 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13016,"thread_start_us":12856,"threads_started":1}
I20260812 06:55:47.491279 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.463440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:39901 (local address 127.0.0.1:37242)
0812 06:55:47.481085 (+ 17645us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.481105 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.481263 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.481818 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.481823 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.481848 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:47.482649 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.482663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.484600 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.484605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.486926 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.486940 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.490411 (+ 3471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.490418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.490600 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.490642 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 06:55:47.490767 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 06:55:47.490899 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17504,"mutex_wait_us":1,"thread_start_us":9873,"threads_started":1}
I20260812 06:55:47.491322 30260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.463805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37242 (local address 127.26.226.3:39901)
0812 06:55:47.480483 (+ 16678us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.480496 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.481296 (+ 800us) server_negotiation.cc:407] Connection header received
0812 06:55:47.481418 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.481425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.481608 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.481822 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:47.482887 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.484452 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.487110 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.490181 (+ 3071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.490350 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.490595 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.490815 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.490980 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16432,"thread_start_us":16178,"threads_started":1}
I20260812 06:55:47.490754 29896 tablet_service.cc:1467] Processing CreateTablet for tablet b06f6fc3c6c54666a5de464e3c23fed9 (DEFAULT_TABLE table=test-table [id=a37dcdb136b34c8c9be9da40b3a44e2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:47.492358 29896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b06f6fc3c6c54666a5de464e3c23fed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:47.492534 29683 tablet_service.cc:1467] Processing CreateTablet for tablet b06f6fc3c6c54666a5de464e3c23fed9 (DEFAULT_TABLE table=test-table [id=a37dcdb136b34c8c9be9da40b3a44e2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:47.493983 29683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b06f6fc3c6c54666a5de464e3c23fed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:47.494436 30117 tablet_service.cc:1467] Processing CreateTablet for tablet b06f6fc3c6c54666a5de464e3c23fed9 (DEFAULT_TABLE table=test-table [id=a37dcdb136b34c8c9be9da40b3a44e2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:47.495518 30117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b06f6fc3c6c54666a5de464e3c23fed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:47.508304 30266 tablet_bootstrap.cc:492] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: Bootstrap starting.
I20260812 06:55:47.509156 30263 tablet_bootstrap.cc:492] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: Bootstrap starting.
I20260812 06:55:47.512194 30266 tablet_bootstrap.cc:654] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:47.512718 30263 tablet_bootstrap.cc:654] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:47.512811 30267 tablet_bootstrap.cc:492] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: Bootstrap starting.
I20260812 06:55:47.513707 30266 log.cc:824] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:47.514431 30263 log.cc:824] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:47.515933 30267 tablet_bootstrap.cc:654] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:47.517025 30267 log.cc:824] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:47.517508 30266 tablet_bootstrap.cc:492] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: No bootstrap required, opened a new log
I20260812 06:55:47.517871 30266 ts_tablet_manager.cc:1397] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260812 06:55:47.526237 30263 tablet_bootstrap.cc:492] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: No bootstrap required, opened a new log
I20260812 06:55:47.526525 30267 tablet_bootstrap.cc:492] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: No bootstrap required, opened a new log
I20260812 06:55:47.526546 30263 ts_tablet_manager.cc:1397] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: Time spent bootstrapping tablet: real 0.020s user 0.015s sys 0.000s
I20260812 06:55:47.526865 30267 ts_tablet_manager.cc:1397] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.006s
I20260812 06:55:47.532388 30266 raft_consensus.cc:348] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.533002 30266 raft_consensus.cc:374] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:47.533129 30266 raft_consensus.cc:729] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b965b78dfba4ebeb071920bfee89363, State: Initialized, Role: FOLLOWER
I20260812 06:55:47.533798 30266 consensus_queue.cc:260] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.536741 30263 raft_consensus.cc:348] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.537112 30263 raft_consensus.cc:374] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:47.537329 30263 raft_consensus.cc:729] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef7c6113d53f41cca4a808b7e620c9f9, State: Initialized, Role: FOLLOWER
I20260812 06:55:47.538381 30011 heartbeater.cc:503] Master 127.26.226.62:41415 was elected leader, sending a full tablet report...
I20260812 06:55:47.538295 30263 consensus_queue.cc:260] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.539065 30266 ts_tablet_manager.cc:1428] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: Time spent starting tablet: real 0.021s user 0.008s sys 0.012s
I20260812 06:55:47.539913 30267 raft_consensus.cc:348] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.540489 30267 raft_consensus.cc:374] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:47.540655 30267 raft_consensus.cc:729] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd6677cf9bdf432088c15d264f78b94d, State: Initialized, Role: FOLLOWER
I20260812 06:55:47.540822 30229 heartbeater.cc:503] Master 127.26.226.62:41415 was elected leader, sending a full tablet report...
I20260812 06:55:47.541497 30263 ts_tablet_manager.cc:1428] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: Time spent starting tablet: real 0.015s user 0.009s sys 0.006s
I20260812 06:55:47.541435 30267 consensus_queue.cc:260] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.545305 30267 ts_tablet_manager.cc:1428] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260812 06:55:47.559108 30271 raft_consensus.cc:482] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:47.559418 30271 raft_consensus.cc:504] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.561450 30271 leader_election.cc:288] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd6677cf9bdf432088c15d264f78b94d (127.26.226.1:41185), ef7c6113d53f41cca4a808b7e620c9f9 (127.26.226.3:39901)
I20260812 06:55:47.570567 30260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.562124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:60313 (local address 127.26.226.3:39901)
0812 06:55:47.562332 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.562339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.563219 (+ 880us) server_negotiation.cc:407] Connection header received
0812 06:55:47.563469 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.563475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.563551 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.563643 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:47.564271 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.565352 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.568138 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.569346 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.569514 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.569588 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.570191 (+ 603us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.570323 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:55:47.570567 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.563468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:45931 (local address 127.26.226.1:41185)
0812 06:55:47.563663 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.563669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.563685 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:55:47.563751 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.563756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.563837 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.563917 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:47.564597 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.565602 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.568137 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.569346 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.569514 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.569588 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.570191 (+ 603us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.570323 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 06:55:47.570845 30279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.561938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:39901 (local address 127.26.226.2:60313)
0812 06:55:47.563103 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.563123 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.563327 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.563676 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.563679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.563705 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:47.564083 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.564101 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.565481 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.565484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.567951 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.567967 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.569643 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.569650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.569892 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.570046 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 06:55:47.570147 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:55:47.570309 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":205,"thread_start_us":824,"threads_started":1}
I20260812 06:55:47.570869 30278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.561726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41185 (local address 127.26.226.2:45931)
0812 06:55:47.562857 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.562877 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.563025 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.563973 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.563979 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.564011 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:47.564456 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.564464 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.565732 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.565737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.567955 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.567967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.569648 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.569653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.569894 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.570046 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 06:55:47.570147 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:55:47.570309 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"mutex_wait_us":206,"thread_start_us":617,"threads_started":1}
I20260812 06:55:47.571920 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06f6fc3c6c54666a5de464e3c23fed9" candidate_uuid: "1b965b78dfba4ebeb071920bfee89363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" is_pre_election: true
I20260812 06:55:47.571920 29710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06f6fc3c6c54666a5de464e3c23fed9" candidate_uuid: "1b965b78dfba4ebeb071920bfee89363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6677cf9bdf432088c15d264f78b94d" is_pre_election: true
W20260812 06:55:47.572335 30012 tablet.cc:2369] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:55:47.572508 30152 raft_consensus.cc:2454] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b965b78dfba4ebeb071920bfee89363 in term 0.
I20260812 06:55:47.572517 29710 raft_consensus.cc:2454] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b965b78dfba4ebeb071920bfee89363 in term 0.
I20260812 06:55:47.573416 29815 leader_election.cc:302] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b965b78dfba4ebeb071920bfee89363, dd6677cf9bdf432088c15d264f78b94d, ef7c6113d53f41cca4a808b7e620c9f9; no voters:
I20260812 06:55:47.573848 30271 raft_consensus.cc:2781] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:47.573992 30271 raft_consensus.cc:482] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:47.574101 30271 raft_consensus.cc:3037] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:47.576326 30271 raft_consensus.cc:504] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.576987 30271 leader_election.cc:288] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [CANDIDATE]: Term 1 election: Requested vote from peers dd6677cf9bdf432088c15d264f78b94d (127.26.226.1:41185), ef7c6113d53f41cca4a808b7e620c9f9 (127.26.226.3:39901)
I20260812 06:55:47.577399 29710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06f6fc3c6c54666a5de464e3c23fed9" candidate_uuid: "1b965b78dfba4ebeb071920bfee89363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6677cf9bdf432088c15d264f78b94d"
I20260812 06:55:47.577419 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b06f6fc3c6c54666a5de464e3c23fed9" candidate_uuid: "1b965b78dfba4ebeb071920bfee89363" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef7c6113d53f41cca4a808b7e620c9f9"
I20260812 06:55:47.577610 30152 raft_consensus.cc:3037] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:47.577608 29710 raft_consensus.cc:3037] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:47.580859 30152 raft_consensus.cc:2454] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b965b78dfba4ebeb071920bfee89363 in term 1.
I20260812 06:55:47.580859 29710 raft_consensus.cc:2454] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b965b78dfba4ebeb071920bfee89363 in term 1.
I20260812 06:55:47.581305 29817 leader_election.cc:302] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b965b78dfba4ebeb071920bfee89363, ef7c6113d53f41cca4a808b7e620c9f9; no voters:
I20260812 06:55:47.581686 30271 raft_consensus.cc:2781] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:47.582360 30271 raft_consensus.cc:686] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 1 LEADER]: Becoming Leader. State: Replica: 1b965b78dfba4ebeb071920bfee89363, State: Running, Role: LEADER
I20260812 06:55:47.582962 30271 consensus_queue.cc:237] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.587488 29448 catalog_manager.cc:5495] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b965b78dfba4ebeb071920bfee89363 (127.26.226.2). New cstate: current_term: 1 leader_uuid: "1b965b78dfba4ebeb071920bfee89363" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:47.622247 30256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.615685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59994 (local address 127.26.226.2:33957)
0812 06:55:47.615891 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.615897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.616041 (+ 144us) server_negotiation.cc:407] Connection header received
0812 06:55:47.616389 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.616394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.616519 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.616678 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:47.617023 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.618113 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.619601 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.620300 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.620437 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.620915 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:47.621593 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:47.621727 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.621869 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.621957 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 06:55:47.628356 30232 tablet.cc:2369] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:55:47.630463 30152 raft_consensus.cc:1261] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 FOLLOWER]: Refusing update from remote peer 1b965b78dfba4ebeb071920bfee89363: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:47.630463 29710 raft_consensus.cc:1261] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d [term 1 FOLLOWER]: Refusing update from remote peer 1b965b78dfba4ebeb071920bfee89363: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:47.631469 30281 consensus_queue.cc:1035] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:47.631837 30271 consensus_queue.cc:1035] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:47.642669 30286 mvcc.cc:205] Tried to move back new op lower bound from 7317576694284156928 to 7317576694103781376. Current Snapshot: MvccSnapshot[applied={T|T < 7317576694284156928}]
I20260812 06:55:47.664148 29446 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:47.664554 29446 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:47.668506 29446 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257: Sending DeleteTablet for 3 replicas of tablet b06f6fc3c6c54666a5de464e3c23fed9
I20260812 06:55:47.670009 29896 tablet_service.cc:1514] Processing DeleteTablet for tablet b06f6fc3c6c54666a5de464e3c23fed9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:47 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:59982
I20260812 06:55:47.670015 29683 tablet_service.cc:1514] Processing DeleteTablet for tablet b06f6fc3c6c54666a5de464e3c23fed9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:47 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:36454
I20260812 06:55:47.671264 30305 tablet_replica.cc:330] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: stopping tablet replica
I20260812 06:55:47.671283 30117 tablet_service.cc:1514] Processing DeleteTablet for tablet b06f6fc3c6c54666a5de464e3c23fed9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:47 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:37242
I20260812 06:55:47.671802 30305 raft_consensus.cc:2229] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:47.672654 30305 raft_consensus.cc:2258] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:47.674031 30307 tablet_replica.cc:330] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: stopping tablet replica
I20260812 06:55:47.674229 29446 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 06:55:47.674829 30307 raft_consensus.cc:2229] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260812 06:55:47.675144 29446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:47.675551 30307 raft_consensus.cc:2258] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:47.678260 30305 ts_tablet_manager.cc:1905] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:47.682272 30307 ts_tablet_manager.cc:1905] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:47.685410 29683 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6066b598df447e8f0a3f5839ff911a (DEFAULT_TABLE table=test-table [id=e0f3af11dfd94781bb5517525f26bab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:47.685433 30116 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6066b598df447e8f0a3f5839ff911a (DEFAULT_TABLE table=test-table [id=e0f3af11dfd94781bb5517525f26bab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:47.686033 29683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6066b598df447e8f0a3f5839ff911a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:47.686026 30116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6066b598df447e8f0a3f5839ff911a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:47.692801 29896 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6066b598df447e8f0a3f5839ff911a (DEFAULT_TABLE table=test-table [id=e0f3af11dfd94781bb5517525f26bab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:47.693336 29896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6066b598df447e8f0a3f5839ff911a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:47.693385 30267 tablet_bootstrap.cc:492] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: Bootstrap starting.
I20260812 06:55:47.695139 30267 tablet_bootstrap.cc:654] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:47.695722 30307 ts_tablet_manager.cc:1918] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:47.695730 30305 ts_tablet_manager.cc:1918] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:47.695905 30307 log.cc:1196] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/b06f6fc3c6c54666a5de464e3c23fed9
I20260812 06:55:47.695906 30305 log.cc:1196] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/b06f6fc3c6c54666a5de464e3c23fed9
I20260812 06:55:47.696535 30307 ts_tablet_manager.cc:1939] T b06f6fc3c6c54666a5de464e3c23fed9 P dd6677cf9bdf432088c15d264f78b94d: Deleting consensus metadata
I20260812 06:55:47.696548 30305 ts_tablet_manager.cc:1939] T b06f6fc3c6c54666a5de464e3c23fed9 P 1b965b78dfba4ebeb071920bfee89363: Deleting consensus metadata
I20260812 06:55:47.698755 29414 catalog_manager.cc:4841] TS 1b965b78dfba4ebeb071920bfee89363 (127.26.226.2:33957): tablet b06f6fc3c6c54666a5de464e3c23fed9 (table test-table [id=a37dcdb136b34c8c9be9da40b3a44e2c]) successfully deleted
I20260812 06:55:47.698725 29415 catalog_manager.cc:4841] TS dd6677cf9bdf432088c15d264f78b94d (127.26.226.1:41185): tablet b06f6fc3c6c54666a5de464e3c23fed9 (table test-table [id=a37dcdb136b34c8c9be9da40b3a44e2c]) successfully deleted
I20260812 06:55:47.699154 30308 tablet_replica.cc:330] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: stopping tablet replica
I20260812 06:55:47.699709 30308 raft_consensus.cc:2229] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:47.700133 30263 tablet_bootstrap.cc:492] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9: Bootstrap starting.
I20260812 06:55:47.700493 30308 raft_consensus.cc:2258] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:47.701963 30263 tablet_bootstrap.cc:654] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:47.702495 30266 tablet_bootstrap.cc:492] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363: Bootstrap starting.
I20260812 06:55:47.703284 30267 tablet_bootstrap.cc:492] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: No bootstrap required, opened a new log
I20260812 06:55:47.703493 30267 ts_tablet_manager.cc:1397] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260812 06:55:47.704432 30267 raft_consensus.cc:348] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.704689 30266 tablet_bootstrap.cc:654] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:47.704727 30267 raft_consensus.cc:374] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:47.704833 30267 raft_consensus.cc:729] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd6677cf9bdf432088c15d264f78b94d, State: Initialized, Role: FOLLOWER
I20260812 06:55:47.705138 30267 consensus_queue.cc:260] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.705745 30267 ts_tablet_manager.cc:1428] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 06:55:47.712191 30308 ts_tablet_manager.cc:1905] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:47.717635 30263 tablet_bootstrap.cc:492] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9: No bootstrap required, opened a new log
I20260812 06:55:47.717660 30266 tablet_bootstrap.cc:492] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363: No bootstrap required, opened a new log
I20260812 06:55:47.717835 30266 ts_tablet_manager.cc:1397] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.007s
I20260812 06:55:47.717908 30263 ts_tablet_manager.cc:1397] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.011s
I20260812 06:55:47.718866 30266 raft_consensus.cc:348] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.718954 30263 raft_consensus.cc:348] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.719208 30263 raft_consensus.cc:374] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:47.719233 30266 raft_consensus.cc:374] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:47.719362 30263 raft_consensus.cc:729] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef7c6113d53f41cca4a808b7e620c9f9, State: Initialized, Role: FOLLOWER
I20260812 06:55:47.719380 30266 raft_consensus.cc:729] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b965b78dfba4ebeb071920bfee89363, State: Initialized, Role: FOLLOWER
I20260812 06:55:47.719730 30266 consensus_queue.cc:260] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.720999 30308 ts_tablet_manager.cc:1918] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:47.721256 30308 log.cc:1196] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/b06f6fc3c6c54666a5de464e3c23fed9
I20260812 06:55:47.721549 30266 ts_tablet_manager.cc:1428] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 06:55:47.720927 30263 consensus_queue.cc:260] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.722206 30263 ts_tablet_manager.cc:1428] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260812 06:55:47.722710 30308 ts_tablet_manager.cc:1939] T b06f6fc3c6c54666a5de464e3c23fed9 P ef7c6113d53f41cca4a808b7e620c9f9: Deleting consensus metadata
I20260812 06:55:47.725618 29421 catalog_manager.cc:4841] TS ef7c6113d53f41cca4a808b7e620c9f9 (127.26.226.3:39901): tablet b06f6fc3c6c54666a5de464e3c23fed9 (table test-table [id=a37dcdb136b34c8c9be9da40b3a44e2c]) successfully deleted
W20260812 06:55:47.755657 29787 tablet.cc:2369] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:55:47.806730 30276 raft_consensus.cc:482] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:47.807013 30276 raft_consensus.cc:504] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.808713 30276 leader_election.cc:288] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd6677cf9bdf432088c15d264f78b94d (127.26.226.1:41185), 1b965b78dfba4ebeb071920bfee89363 (127.26.226.2:33957)
I20260812 06:55:47.815356 30056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.808969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41185 (local address 127.26.226.3:40713)
0812 06:55:47.809247 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.809274 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.809428 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.809978 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.809984 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.810019 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:47.810300 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.810312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.811790 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.811795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.813361 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.813374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.814530 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.814536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.814758 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.814795 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 06:55:47.814899 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 06:55:47.815047 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":39}
I20260812 06:55:47.815407 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.809409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:40713 (local address 127.26.226.1:41185)
0812 06:55:47.809593 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.809599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.809619 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:47.809700 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.809705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.809787 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.809885 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:47.810479 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.811634 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.813528 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.814353 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.814537 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.814613 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.814958 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.815107 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:55:47.816406 29710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6066b598df447e8f0a3f5839ff911a" candidate_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6677cf9bdf432088c15d264f78b94d" is_pre_election: true
I20260812 06:55:47.816680 29710 raft_consensus.cc:2454] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef7c6113d53f41cca4a808b7e620c9f9 in term 0.
I20260812 06:55:47.817193 30256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.810184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:36767 (local address 127.26.226.2:33957)
0812 06:55:47.810419 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.810427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.810447 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:47.810512 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.810516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.810576 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.810694 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:47.811197 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.812138 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.814763 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.816109 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.816265 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.816330 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.816718 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.816859 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 06:55:47.817229 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.809085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33957 (local address 127.26.226.3:36767)
0812 06:55:47.809963 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.809980 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.810092 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.810704 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.810708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.810732 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:47.811041 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.811049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.812281 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.812287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.814565 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.814584 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.816334 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.816341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.816523 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.816562 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 06:55:47.816669 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 06:55:47.816799 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":27,"thread_start_us":605,"threads_started":1}
I20260812 06:55:47.817369 30045 leader_election.cc:302] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd6677cf9bdf432088c15d264f78b94d, ef7c6113d53f41cca4a808b7e620c9f9; no voters:
I20260812 06:55:47.817823 30276 raft_consensus.cc:2781] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:47.817941 30276 raft_consensus.cc:482] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:47.818024 30276 raft_consensus.cc:3037] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:47.818405 29927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6066b598df447e8f0a3f5839ff911a" candidate_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b965b78dfba4ebeb071920bfee89363" is_pre_election: true
I20260812 06:55:47.818954 29927 raft_consensus.cc:2454] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef7c6113d53f41cca4a808b7e620c9f9 in term 0.
I20260812 06:55:47.821416 30276 raft_consensus.cc:504] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.822345 30276 leader_election.cc:288] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [CANDIDATE]: Term 1 election: Requested vote from peers dd6677cf9bdf432088c15d264f78b94d (127.26.226.1:41185), 1b965b78dfba4ebeb071920bfee89363 (127.26.226.2:33957)
I20260812 06:55:47.822530 29710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6066b598df447e8f0a3f5839ff911a" candidate_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6677cf9bdf432088c15d264f78b94d"
I20260812 06:55:47.822796 29710 raft_consensus.cc:3037] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:47.825893 29710 raft_consensus.cc:2454] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef7c6113d53f41cca4a808b7e620c9f9 in term 1.
I20260812 06:55:47.826272 30045 leader_election.cc:302] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd6677cf9bdf432088c15d264f78b94d, ef7c6113d53f41cca4a808b7e620c9f9; no voters:
I20260812 06:55:47.826397 29927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6066b598df447e8f0a3f5839ff911a" candidate_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b965b78dfba4ebeb071920bfee89363"
I20260812 06:55:47.826610 29927 raft_consensus.cc:3037] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:47.827172 30276 raft_consensus.cc:2781] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:47.830365 29927 raft_consensus.cc:2454] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef7c6113d53f41cca4a808b7e620c9f9 in term 1.
I20260812 06:55:47.831233 30276 raft_consensus.cc:686] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 LEADER]: Becoming Leader. State: Replica: ef7c6113d53f41cca4a808b7e620c9f9, State: Running, Role: LEADER
I20260812 06:55:47.831748 30276 consensus_queue.cc:237] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.835999 29448 catalog_manager.cc:5495] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef7c6113d53f41cca4a808b7e620c9f9 (127.26.226.3). New cstate: current_term: 1 leader_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } health_report { overall_health: HEALTHY } } }
Aug 12 06:55:47 dist-test-slave-t3q3 krb5kdc[29358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517745, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:47.898634 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.882724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34264 (local address 127.26.226.62:41415)
0812 06:55:47.882969 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.882978 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.883628 (+ 650us) server_negotiation.cc:407] Connection header received
0812 06:55:47.883856 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.883860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.883932 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.884013 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:47.885665 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.886352 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.888274 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.888677 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.888783 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.892889 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:47.892894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:47.892899 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:47.892937 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:47.895577 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:47.896476 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:47.896480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:47.896482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:47.896616 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:47.897075 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:47.897079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:47.897080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:47.897344 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:47.897480 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:47.898011 (+ 531us) server_negotiation.cc:299] Negotiation successful
0812 06:55:47.898371 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":75}
I20260812 06:55:47.898725 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.882754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:41415 (local address 127.0.0.1:34264)
0812 06:55:47.883498 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.883521 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.883691 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.884118 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.884123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.885114 (+ 991us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:47.885507 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.885522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.886496 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.886502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.888131 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.888149 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.888880 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.888886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.889057 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.889829 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:47.889854 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:47.892680 (+ 2826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:47.895725 (+ 3045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:47.895737 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:47.895745 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:47.896353 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:47.896738 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:47.896743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:47.896746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:47.896957 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:47.897481 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:47.897489 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:47.897685 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 06:55:47.897901 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 06:55:47.898222 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":386,"threads_started":1}
I20260812 06:55:47.904213 29448 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:34264:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 06:55:47.905340 29448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:47.912410 29683 tablet_service.cc:1467] Processing CreateTablet for tablet e7d8d66acae5478a89d7a190c130e3b9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f1b438a464464d44b0c03c01a69a35bc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:55:47.913043 29683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d8d66acae5478a89d7a190c130e3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:47.912848 29896 tablet_service.cc:1467] Processing CreateTablet for tablet e7d8d66acae5478a89d7a190c130e3b9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f1b438a464464d44b0c03c01a69a35bc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:55:47.913442 29896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d8d66acae5478a89d7a190c130e3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:47.913950 30117 tablet_service.cc:1467] Processing CreateTablet for tablet e7d8d66acae5478a89d7a190c130e3b9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f1b438a464464d44b0c03c01a69a35bc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:55:47.914489 30117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d8d66acae5478a89d7a190c130e3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:47.920933 30267 tablet_bootstrap.cc:492] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: Bootstrap starting.
I20260812 06:55:47.921170 30266 tablet_bootstrap.cc:492] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: Bootstrap starting.
I20260812 06:55:47.921612 30263 tablet_bootstrap.cc:492] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: Bootstrap starting.
I20260812 06:55:47.922829 30267 tablet_bootstrap.cc:654] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:47.922892 30266 tablet_bootstrap.cc:654] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:47.923408 30263 tablet_bootstrap.cc:654] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:47.932808 30267 tablet_bootstrap.cc:492] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: No bootstrap required, opened a new log
I20260812 06:55:47.933089 30267 ts_tablet_manager.cc:1397] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260812 06:55:47.934012 30267 raft_consensus.cc:348] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.934350 30266 tablet_bootstrap.cc:492] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: No bootstrap required, opened a new log
I20260812 06:55:47.934373 30267 raft_consensus.cc:374] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:47.934465 30267 raft_consensus.cc:729] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd6677cf9bdf432088c15d264f78b94d, State: Initialized, Role: FOLLOWER
I20260812 06:55:47.934832 30267 consensus_queue.cc:260] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.935003 30266 ts_tablet_manager.cc:1397] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260812 06:55:47.935539 30277 tablet_replica.cc:439] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } } }
I20260812 06:55:47.935930 30277 tablet_replica.cc:442] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:47.935828 30266 raft_consensus.cc:348] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.936117 30266 raft_consensus.cc:374] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:47.936235 30266 raft_consensus.cc:729] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b965b78dfba4ebeb071920bfee89363, State: Initialized, Role: FOLLOWER
I20260812 06:55:47.936547 30266 consensus_queue.cc:260] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.935549 30267 ts_tablet_manager.cc:1428] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 06:55:47.937059 30294 tablet_replica.cc:439] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } } }
I20260812 06:55:47.937450 30294 tablet_replica.cc:442] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:47.937947 30266 ts_tablet_manager.cc:1428] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:47.942309 30263 tablet_bootstrap.cc:492] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: No bootstrap required, opened a new log
I20260812 06:55:47.942497 30263 ts_tablet_manager.cc:1397] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.005s
I20260812 06:55:47.943276 30263 raft_consensus.cc:348] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.943517 30263 raft_consensus.cc:374] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:47.943619 30263 raft_consensus.cc:729] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef7c6113d53f41cca4a808b7e620c9f9, State: Initialized, Role: FOLLOWER
I20260812 06:55:47.944132 30263 consensus_queue.cc:260] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:47.944546 30276 tablet_replica.cc:439] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } } }
I20260812 06:55:47.944829 30276 tablet_replica.cc:442] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:47.945240 30263 ts_tablet_manager.cc:1428] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:48.113906 30276 raft_consensus.cc:482] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:48.114166 30276 raft_consensus.cc:504] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:48.114886 30276 leader_election.cc:288] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd6677cf9bdf432088c15d264f78b94d (127.26.226.1:41185), 1b965b78dfba4ebeb071920bfee89363 (127.26.226.2:33957)
I20260812 06:55:48.115480 29710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7d8d66acae5478a89d7a190c130e3b9" candidate_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6677cf9bdf432088c15d264f78b94d" is_pre_election: true
I20260812 06:55:48.115537 29927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7d8d66acae5478a89d7a190c130e3b9" candidate_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b965b78dfba4ebeb071920bfee89363" is_pre_election: true
I20260812 06:55:48.115773 29710 raft_consensus.cc:2454] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef7c6113d53f41cca4a808b7e620c9f9 in term 0.
I20260812 06:55:48.115856 29927 raft_consensus.cc:2454] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef7c6113d53f41cca4a808b7e620c9f9 in term 0.
I20260812 06:55:48.116230 30045 leader_election.cc:302] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd6677cf9bdf432088c15d264f78b94d, ef7c6113d53f41cca4a808b7e620c9f9; no voters:
I20260812 06:55:48.116575 30276 raft_consensus.cc:2781] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:48.116689 30276 raft_consensus.cc:482] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:48.116801 30276 raft_consensus.cc:3037] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:48.120007 30276 raft_consensus.cc:504] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:48.120847 30276 leader_election.cc:288] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [CANDIDATE]: Term 1 election: Requested vote from peers dd6677cf9bdf432088c15d264f78b94d (127.26.226.1:41185), 1b965b78dfba4ebeb071920bfee89363 (127.26.226.2:33957)
I20260812 06:55:48.121423 29710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7d8d66acae5478a89d7a190c130e3b9" candidate_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6677cf9bdf432088c15d264f78b94d"
I20260812 06:55:48.121470 29927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7d8d66acae5478a89d7a190c130e3b9" candidate_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b965b78dfba4ebeb071920bfee89363"
I20260812 06:55:48.121702 29710 raft_consensus.cc:3037] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:48.121718 29927 raft_consensus.cc:3037] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:48.124856 29927 raft_consensus.cc:2454] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef7c6113d53f41cca4a808b7e620c9f9 in term 1.
I20260812 06:55:48.124848 29710 raft_consensus.cc:2454] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef7c6113d53f41cca4a808b7e620c9f9 in term 1.
I20260812 06:55:48.125339 30044 leader_election.cc:302] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b965b78dfba4ebeb071920bfee89363, ef7c6113d53f41cca4a808b7e620c9f9; no voters:
I20260812 06:55:48.125725 30276 raft_consensus.cc:2781] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:48.125893 30276 raft_consensus.cc:686] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 LEADER]: Becoming Leader. State: Replica: ef7c6113d53f41cca4a808b7e620c9f9, State: Running, Role: LEADER
I20260812 06:55:48.126166 30276 consensus_queue.cc:237] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } }
I20260812 06:55:48.127930 30320 tablet_replica.cc:439] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: TxnStatusTablet state changed. Reason: New leader ef7c6113d53f41cca4a808b7e620c9f9. Latest consensus state: current_term: 1 leader_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } } }
I20260812 06:55:48.128293 30320 tablet_replica.cc:442] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:55:48.129568 29447 catalog_manager.cc:5495] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef7c6113d53f41cca4a808b7e620c9f9 (127.26.226.3). New cstate: current_term: 1 leader_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } health_report { overall_health: HEALTHY } } }
I20260812 06:55:48.133530 30337 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 06:55:48.133873 30337 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 06:55:48.184573 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.177483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:39901 (local address 127.0.0.1:37248)
0812 06:55:48.177761 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:48.177785 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:48.177966 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:48.178719 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:48.178723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:48.178733 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:48.178912 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:48.178919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.180526 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.180531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:48.181557 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:48.181568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.182640 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.182645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:48.182813 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.182879 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:48.184048 (+ 1169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:48.184112 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 06:55:48.184197 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 06:55:48.184331 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 06:55:48.184715 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.177625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37248 (local address 127.26.226.3:39901)
0812 06:55:48.178223 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.178229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.178250 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:55:48.178368 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.178374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.178499 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.178686 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:48.179086 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.180391 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.181707 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.182430 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.182567 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.183090 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:48.183928 (+ 838us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:48.184122 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:48.184230 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 06:55:48.184324 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":271,"threads_started":1}
I20260812 06:55:48.189925 29710 raft_consensus.cc:1261] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d [term 1 FOLLOWER]: Refusing update from remote peer ef7c6113d53f41cca4a808b7e620c9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:48.190070 29927 raft_consensus.cc:1261] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363 [term 1 FOLLOWER]: Refusing update from remote peer ef7c6113d53f41cca4a808b7e620c9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:48.190846 30276 consensus_queue.cc:1035] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:48.191596 30276 consensus_queue.cc:1035] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:48.199997 30294 tablet_replica.cc:439] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: TxnStatusTablet state changed. Reason: New leader ef7c6113d53f41cca4a808b7e620c9f9. Latest consensus state: current_term: 1 leader_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } } }
I20260812 06:55:48.200364 30294 tablet_replica.cc:442] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:48.201790 30277 tablet_replica.cc:439] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: TxnStatusTablet state changed. Reason: New leader ef7c6113d53f41cca4a808b7e620c9f9. Latest consensus state: current_term: 1 leader_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } } }
I20260812 06:55:48.202109 30277 tablet_replica.cc:442] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:48.205221 30344 mvcc.cc:205] Tried to move back new op lower bound from 7317576696578002944 to 7317576696327127040. Current Snapshot: MvccSnapshot[applied={T|T < 7317576696578002944}]
I20260812 06:55:48.225050 30276 tablet_replica.cc:439] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } } }
I20260812 06:55:48.226063 30276 tablet_replica.cc:442] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:55:48.237159 30276 tablet_replica.cc:439] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } } }
I20260812 06:55:48.237547 30276 tablet_replica.cc:442] T e7d8d66acae5478a89d7a190c130e3b9 P ef7c6113d53f41cca4a808b7e620c9f9: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:55:48.244764 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.230042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37258 (local address 127.26.226.3:39901)
0812 06:55:48.230232 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.230240 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.232614 (+ 2374us) server_negotiation.cc:407] Connection header received
0812 06:55:48.233489 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.233495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.233595 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.233661 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:48.237878 (+ 4217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.239046 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.241252 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.241942 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.242043 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.243771 (+ 1728us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:48.243919 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:48.243996 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:48.244264 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 06:55:48.244354 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:55:48.243899 30277 tablet_replica.cc:439] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } } }
I20260812 06:55:48.245539 30277 tablet_replica.cc:442] T e7d8d66acae5478a89d7a190c130e3b9 P dd6677cf9bdf432088c15d264f78b94d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:48.246992 30294 tablet_replica.cc:439] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 } } peers { permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 } } peers { permanent_uuid: "ef7c6113d53f41cca4a808b7e620c9f9" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39901 } } }
I20260812 06:55:48.247267 30294 tablet_replica.cc:442] T e7d8d66acae5478a89d7a190c130e3b9 P 1b965b78dfba4ebeb071920bfee89363: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:48.271521 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.258529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:57431 (local address 127.26.226.3:39901)
0812 06:55:48.258723 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.258730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.264842 (+ 6112us) server_negotiation.cc:407] Connection header received
0812 06:55:48.265072 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.265079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.265199 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.265294 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:48.265917 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.267084 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.269347 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.270007 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.270134 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.270688 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:48.270818 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:48.270906 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:48.271060 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 06:55:48.271150 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":80}
I20260812 06:55:48.271644 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.258414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:39901 (local address 127.26.226.3:57431)
0812 06:55:48.264709 (+ 6295us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:48.264732 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:48.264900 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:48.265336 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:48.265341 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:48.265354 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:48.265753 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:48.265766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.267226 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.267232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:48.269179 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:48.269196 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.270221 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.270228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:48.270416 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.270490 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:48.270929 (+ 439us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:48.270941 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 06:55:48.271027 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 06:55:48.271184 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6064,"mutex_wait_us":1,"thread_start_us":5909,"threads_started":1}
I20260812 06:55:48.283406 29710 raft_consensus.cc:1261] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d [term 1 FOLLOWER]: Refusing update from remote peer ef7c6113d53f41cca4a808b7e620c9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:48.283444 29925 raft_consensus.cc:1261] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363 [term 1 FOLLOWER]: Refusing update from remote peer ef7c6113d53f41cca4a808b7e620c9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:48.284104 30276 consensus_queue.cc:1035] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b965b78dfba4ebeb071920bfee89363" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:48.284623 30354 consensus_queue.cc:1035] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd6677cf9bdf432088c15d264f78b94d" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:48.359294 29446 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:48.359597 29446 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:48.363204 29446 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23a7eb92554e4a2f9478b4445016c257: Sending DeleteTablet for 3 replicas of tablet 1f6066b598df447e8f0a3f5839ff911a
I20260812 06:55:48.364151 29683 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f6066b598df447e8f0a3f5839ff911a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:48 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:36454
I20260812 06:55:48.364221 29896 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f6066b598df447e8f0a3f5839ff911a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:48 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:59982
I20260812 06:55:48.364573 30117 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f6066b598df447e8f0a3f5839ff911a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:48 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:37242
I20260812 06:55:48.375306 30368 tablet_replica.cc:330] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: stopping tablet replica
I20260812 06:55:48.375336 30370 tablet_replica.cc:330] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363: stopping tablet replica
I20260812 06:55:48.375622 30368 raft_consensus.cc:2229] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:48.375559 30370 raft_consensus.cc:2229] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:48.376009 30368 raft_consensus.cc:2258] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:48.376147 30370 raft_consensus.cc:2258] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:48.377529 30368 ts_tablet_manager.cc:1905] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:48.377564 30370 ts_tablet_manager.cc:1905] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:48.385403 30368 ts_tablet_manager.cc:1918] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:48.385492 30370 ts_tablet_manager.cc:1918] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:48.385627 30368 log.cc:1196] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/1f6066b598df447e8f0a3f5839ff911a
I20260812 06:55:48.385744 30370 log.cc:1196] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/1f6066b598df447e8f0a3f5839ff911a
I20260812 06:55:48.386166 30368 ts_tablet_manager.cc:1939] T 1f6066b598df447e8f0a3f5839ff911a P dd6677cf9bdf432088c15d264f78b94d: Deleting consensus metadata
I20260812 06:55:48.386369 30370 ts_tablet_manager.cc:1939] T 1f6066b598df447e8f0a3f5839ff911a P 1b965b78dfba4ebeb071920bfee89363: Deleting consensus metadata
I20260812 06:55:48.387437 29415 catalog_manager.cc:4841] TS dd6677cf9bdf432088c15d264f78b94d (127.26.226.1:41185): tablet 1f6066b598df447e8f0a3f5839ff911a (table test-table [id=e0f3af11dfd94781bb5517525f26bab4]) successfully deleted
I20260812 06:55:48.387656 29414 catalog_manager.cc:4841] TS 1b965b78dfba4ebeb071920bfee89363 (127.26.226.2:33957): tablet 1f6066b598df447e8f0a3f5839ff911a (table test-table [id=e0f3af11dfd94781bb5517525f26bab4]) successfully deleted
I20260812 06:55:48.388651 30373 tablet_replica.cc:330] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9: stopping tablet replica
I20260812 06:55:48.389027 30373 raft_consensus.cc:2229] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:48.389911 30373 raft_consensus.cc:2258] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:48.391767 30373 ts_tablet_manager.cc:1905] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:48.402274 30373 ts_tablet_manager.cc:1918] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:48.402540 30373 log.cc:1196] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/1f6066b598df447e8f0a3f5839ff911a
I20260812 06:55:48.403282 30373 ts_tablet_manager.cc:1939] T 1f6066b598df447e8f0a3f5839ff911a P ef7c6113d53f41cca4a808b7e620c9f9: Deleting consensus metadata
I20260812 06:55:48.404826 29421 catalog_manager.cc:4841] TS ef7c6113d53f41cca4a808b7e620c9f9 (127.26.226.3:39901): tablet 1f6066b598df447e8f0a3f5839ff911a (table test-table [id=e0f3af11dfd94781bb5517525f26bab4]) successfully deleted
Aug 12 06:55:48 dist-test-slave-t3q3 krb5kdc[29358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517747, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:55:48.423992 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.391444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36456 (local address 127.26.226.1:41185)
0812 06:55:48.408573 (+ 17129us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.408578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.408596 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:55:48.408651 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.408655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.408715 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.408858 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:48.410459 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.411299 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.412707 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.413155 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.413304 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.417476 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:48.417492 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:48.417507 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:48.417550 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:48.420839 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:48.421564 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:48.421573 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:48.421575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:48.421713 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:48.422121 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:48.422125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:48.422127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:48.422519 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:48.422836 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:48.423157 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 06:55:48.423394 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17008,"thread_start_us":16858,"threads_started":1}
W20260812 06:55:48.424903 29779 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36456
I20260812 06:55:48.447010 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.428331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34272 (local address 127.26.226.62:41415)
0812 06:55:48.432115 (+ 3784us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.432122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.432602 (+ 480us) server_negotiation.cc:407] Connection header received
0812 06:55:48.432733 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.432740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.432824 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.432950 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:48.434356 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.435471 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.436844 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.437513 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.437657 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.440099 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:48.440107 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:48.440112 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:48.440152 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:48.443859 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:48.444710 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:48.444718 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:48.444722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:48.444886 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:48.445317 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:48.445323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:48.445327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:48.445733 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:48.446004 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:48.446331 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 06:55:48.446588 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3608,"thread_start_us":3481,"threads_started":1}
W20260812 06:55:48.447722 29446 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34272
I20260812 06:55:48.450088 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29559
I20260812 06:55:48.466921 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.450988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36466 (local address 127.26.226.1:41185)
0812 06:55:48.451256 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.451263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.452583 (+ 1320us) server_negotiation.cc:407] Connection header received
0812 06:55:48.452819 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.452825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.452907 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.452997 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:48.454658 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.455371 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.457177 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.457573 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.457670 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.460725 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:48.460730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:48.460734 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:48.460805 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:48.464118 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:48.464973 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:48.464977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:48.464980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:48.465122 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:48.465542 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:48.465546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:48.465548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:48.465819 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:48.466023 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:48.466437 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 06:55:48.466612 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
W20260812 06:55:48.467533 29779 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36466
W20260812 06:55:48.468192 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29559: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:55:48.468413 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 29559
I20260812 06:55:48.494704 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29790
Aug 12 06:55:48 dist-test-slave-t3q3 krb5kdc[29358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517747, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:55:48.512879 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.495594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.26.226.2:33957)
0812 06:55:48.496172 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.496177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.496195 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:55:48.496258 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.496262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.496334 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.496427 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:48.498082 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.498795 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.500952 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.501684 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.501831 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.506426 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:48.506443 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:48.506460 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:48.506498 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:48.509605 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:48.510568 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:48.510576 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:48.510578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:48.510756 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:48.511214 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:48.511218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:48.511220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:48.511602 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:48.511832 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:48.512309 (+ 477us) server_negotiation.cc:299] Negotiation successful
0812 06:55:48.512503 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":306,"threads_started":1}
W20260812 06:55:48.513739 29998 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59998
W20260812 06:55:48.514536 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29790: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:55:48.514735 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 29790
I20260812 06:55:48.547873 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30017
Aug 12 06:55:48 dist-test-slave-t3q3 krb5kdc[29358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517747, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:55:48.567899 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.548873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37274 (local address 127.26.226.3:39901)
0812 06:55:48.549108 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.549115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.549139 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:55:48.549302 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.549308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.549405 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.549516 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:48.550801 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.552009 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.553565 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.554304 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.554456 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.559010 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:48.559029 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:48.559050 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:48.559104 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:48.563520 (+ 4416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:48.564556 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:48.564569 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:48.564573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:48.564802 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:48.565351 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:48.565359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:48.565363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:48.566065 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:48.566461 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:48.567048 (+ 587us) server_negotiation.cc:299] Negotiation successful
0812 06:55:48.567479 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260812 06:55:48.568673 30218 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37274
W20260812 06:55:48.569384 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30017: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:55:48.569545 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 30017
I20260812 06:55:48.603816 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29380
I20260812 06:55:48.605139 29505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:48.724260 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 29380
2026-08-12T06:55:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5144 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:48 dist-test-slave-t3q3 krb5kdc[30392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:48 dist-test-slave-t3q3 krb5kdc[30392](info): set up 2 sockets
Aug 12 06:55:48 dist-test-slave-t3q3 krb5kdc[30392](info): commencing operation
krb5kdc: starting...
W20260812 06:55:50.065440 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:55:50 dist-test-slave-t3q3 krb5kdc[30392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:55:50.285375 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:45859
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:45859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:50.621974 30418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:50.622331 30418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:50.622500 30418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:50.629743 30418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:50.629881 30418 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 06:55:50.629979 30418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:50.630110 30418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:50.630198 30418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:50.630290 30418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:50.640136 30418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:45859
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:45859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:50.641438 30418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:50.642726 30418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:50.655193 30426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:50.655160 30428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:50.655159 30430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:50.656437 30418 server_base.cc:1034] running on GCE node
I20260812 06:55:50.657243 30418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:50.658636 30418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:50.659870 30418 hybrid_clock.cc:648] HybridClock initialized: now 1786517750659795 us; error 56 us; skew 500 ppm
I20260812 06:55:50.660264 30418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:50 dist-test-slave-t3q3 krb5kdc[30392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517750, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:50.663574 30418 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:55:50.665072 30418 webserver.cc:460] Webserver started at http://127.26.226.62:36019/ using document root <none> and password file <none>
I20260812 06:55:50.665900 30418 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:50.666021 30418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:50.666378 30418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:50.669587 30418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "3bb8020277904ae7907b7bf4e6bb8abb"
format_stamp: "Formatted at 2026-08-12 06:55:50 on dist-test-slave-t3q3"
server_key: "e9f2a77033dcb5302ce572198c45d38d"
server_key_iv: "35ec2e0bee4a8255c91bcc4674eb8403"
server_key_version: "encryptionkey@0"
I20260812 06:55:50.670418 30418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "3bb8020277904ae7907b7bf4e6bb8abb"
format_stamp: "Formatted at 2026-08-12 06:55:50 on dist-test-slave-t3q3"
server_key: "e9f2a77033dcb5302ce572198c45d38d"
server_key_iv: "35ec2e0bee4a8255c91bcc4674eb8403"
server_key_version: "encryptionkey@0"
I20260812 06:55:50.675927 30418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260812 06:55:50.679596 30441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:50.681010 30418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:50.681212 30418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "3bb8020277904ae7907b7bf4e6bb8abb"
format_stamp: "Formatted at 2026-08-12 06:55:50 on dist-test-slave-t3q3"
server_key: "e9f2a77033dcb5302ce572198c45d38d"
server_key_iv: "35ec2e0bee4a8255c91bcc4674eb8403"
server_key_version: "encryptionkey@0"
I20260812 06:55:50.681439 30418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:50.700945 30418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:50.705166 30418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:50.705459 30418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:50.727985 30418 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:45859
I20260812 06:55:50.728005 30539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:45859 every 8 connection(s)
I20260812 06:55:50.729419 30418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:50.737399 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 30418
I20260812 06:55:50.737619 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:55:50.737897 30540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:50.738372 27528 external_mini_cluster.cc:1428] Setting key c3d88d5a19f69f1a06cf5833a66ff9a7
I20260812 06:55:50.752668 30540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb: Bootstrap starting.
Aug 12 06:55:50 dist-test-slave-t3q3 krb5kdc[30392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:50.756712 30540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:50.758162 30540 log.cc:824] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:50.761984 30540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb: No bootstrap required, opened a new log
I20260812 06:55:50.763782 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:50.740410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41370 (local address 127.26.226.62:45859)
0812 06:55:50.741349 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:50.741366 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:50.741420 (+ 54us) server_negotiation.cc:407] Connection header received
0812 06:55:50.743252 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:50.743281 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:50.743768 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:50.744268 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:50.745562 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.747513 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.748882 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.750013 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.750214 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:50.754396 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:50.754408 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:50.754428 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:50.754480 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:50.758680 (+ 4200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:50.759752 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:50.759762 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:50.759770 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:50.760000 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:50.760506 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:50.760511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:50.760514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:50.761265 (+ 751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:50.761571 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:50.761972 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 06:55:50.762438 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":407,"threads_started":1}
I20260812 06:55:50.770478 30540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb8020277904ae7907b7bf4e6bb8abb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45859 } }
I20260812 06:55:50.770825 30540 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:50.770900 30540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb8020277904ae7907b7bf4e6bb8abb, State: Initialized, Role: FOLLOWER
I20260812 06:55:50.771517 30540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb8020277904ae7907b7bf4e6bb8abb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45859 } }
I20260812 06:55:50.771837 30540 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:50.772004 30540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:50.772238 30540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:50.774446 30540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb8020277904ae7907b7bf4e6bb8abb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45859 } }
I20260812 06:55:50.774998 30540 leader_election.cc:302] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bb8020277904ae7907b7bf4e6bb8abb; no voters:
I20260812 06:55:50.775625 30540 leader_election.cc:288] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:50.775810 30547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:50.776881 30547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [term 1 LEADER]: Becoming Leader. State: Replica: 3bb8020277904ae7907b7bf4e6bb8abb, State: Running, Role: LEADER
I20260812 06:55:50.777483 30547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb8020277904ae7907b7bf4e6bb8abb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45859 } }
I20260812 06:55:50.778044 30540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:50.780480 30553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bb8020277904ae7907b7bf4e6bb8abb. Latest consensus state: current_term: 1 leader_uuid: "3bb8020277904ae7907b7bf4e6bb8abb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb8020277904ae7907b7bf4e6bb8abb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45859 } } }
I20260812 06:55:50.780894 30553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:50.786191 30551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bb8020277904ae7907b7bf4e6bb8abb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb8020277904ae7907b7bf4e6bb8abb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45859 } } }
I20260812 06:55:50.786444 30551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:50.789156 30560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:50.793620 30560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:50.802031 30560 catalog_manager.cc:1347] Generated new cluster ID: 7f5964ed74ba46219f432e336f70d0ab
I20260812 06:55:50.802155 30560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:50.812466 30560 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:50.813776 30560 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:50.824731 30560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb: Generated new TSK 0
I20260812 06:55:50.825623 30560 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:55:50.892021 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:45859
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:51.237741 30587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:51.238075 30587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:51.238240 30587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:51.246644 30587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:51.246779 30587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:51.246973 30587 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 06:55:51.247071 30587 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:55:51.247217 30587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:55:51.258882 30587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.226.62:45859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:51.260430 30587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:51.261814 30587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:51.275051 30598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:51.275154 30595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:51.275118 30594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:51.276181 30587 server_base.cc:1034] running on GCE node
I20260812 06:55:51.276859 30587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:51.277832 30587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:51.279079 30587 hybrid_clock.cc:648] HybridClock initialized: now 1786517751279017 us; error 48 us; skew 500 ppm
I20260812 06:55:51.279477 30587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:51 dist-test-slave-t3q3 krb5kdc[30392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517751, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:51.286095 30587 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:55:51.287734 30587 webserver.cc:460] Webserver started at http://127.26.226.1:44543/ using document root <none> and password file <none>
I20260812 06:55:51.288527 30587 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:51.288689 30587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:51.289040 30587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:51.292308 30587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "efd7cbf9aeb0417eb8f3895be54521a6"
format_stamp: "Formatted at 2026-08-12 06:55:51 on dist-test-slave-t3q3"
server_key: "e79315f169130d3906d9ac60650ff2ca"
server_key_iv: "8c6b9ce400645d879e18d96b511556f6"
server_key_version: "encryptionkey@0"
I20260812 06:55:51.293143 30587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "efd7cbf9aeb0417eb8f3895be54521a6"
format_stamp: "Formatted at 2026-08-12 06:55:51 on dist-test-slave-t3q3"
server_key: "e79315f169130d3906d9ac60650ff2ca"
server_key_iv: "8c6b9ce400645d879e18d96b511556f6"
server_key_version: "encryptionkey@0"
I20260812 06:55:51.298439 30587 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:55:51.301935 30607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:51.303297 30587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:55:51.303491 30587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "efd7cbf9aeb0417eb8f3895be54521a6"
format_stamp: "Formatted at 2026-08-12 06:55:51 on dist-test-slave-t3q3"
server_key: "e79315f169130d3906d9ac60650ff2ca"
server_key_iv: "8c6b9ce400645d879e18d96b511556f6"
server_key_version: "encryptionkey@0"
I20260812 06:55:51.303726 30587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:51.318606 30587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:51.322883 30587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:51.323215 30587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:51.326306 30587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:51.326452 30587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:51.326586 30587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:51.326710 30587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:55:51 dist-test-slave-t3q3 krb5kdc[30392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517751, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:51.352687 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.327541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:45859 (local address 127.26.226.1:55295)
0812 06:55:51.328528 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:51.328589 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:51.331389 (+ 2800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:51.332403 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:51.332421 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:51.333278 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:51.334209 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.334266 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.336020 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.336029 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.337412 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.337426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.338689 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.338694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.338874 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.339670 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:51.339705 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:51.343307 (+ 3602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:51.347213 (+ 3906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:51.347248 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:51.347270 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:51.347970 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:51.348491 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:51.348499 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:51.348504 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:51.348807 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:51.349542 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:51.349551 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:51.350089 (+ 538us) client_negotiation.cc:769] Sending connection context
0812 06:55:51.350493 (+ 404us) client_negotiation.cc:240] Negotiation successful
0812 06:55:51.351102 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":445,"threads_started":1}
I20260812 06:55:51.353173 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.326842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:55295 (local address 127.26.226.62:45859)
0812 06:55:51.327635 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:51.327642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:51.328732 (+ 1090us) server_negotiation.cc:407] Connection header received
0812 06:55:51.331749 (+ 3017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:51.331755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:51.331848 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:51.331970 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:51.334494 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.335790 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.337566 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.338457 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.338632 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.343554 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:51.343559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:51.343563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:51.343597 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:51.347053 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:51.348149 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:51.348154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:51.348158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:51.348327 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:51.348945 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:51.348951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:51.348956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:51.349378 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:51.349542 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:51.352306 (+ 2764us) server_negotiation.cc:299] Negotiation successful
0812 06:55:51.352624 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":546,"threads_started":1}
I20260812 06:55:51.380553 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.363560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:43809 (local address 127.26.226.62:45859)
0812 06:55:51.363728 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:51.363732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:51.364608 (+ 876us) server_negotiation.cc:407] Connection header received
0812 06:55:51.364890 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:51.364903 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:51.364978 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:51.365098 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:51.366854 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.367628 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.369767 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.370503 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.370683 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.373590 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:51.373595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:51.373599 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:51.373635 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:51.377523 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:51.378336 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:51.378342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:51.378346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:51.378491 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:51.378906 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:51.378912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:51.378915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:51.379278 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:51.379456 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:51.379886 (+ 430us) server_negotiation.cc:299] Negotiation successful
0812 06:55:51.380154 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:55:51.380707 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.363651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:45859 (local address 127.26.226.1:43809)
0812 06:55:51.364468 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:51.364489 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:51.364699 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:51.365138 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:51.365147 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:51.366079 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:51.366675 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.366694 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.367786 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.367792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.369587 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.369601 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.370952 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.370957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.371230 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.372206 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:51.372244 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:51.373356 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:51.377669 (+ 4313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:51.377675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:51.377680 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:51.378175 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:51.378619 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:51.378624 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:51.378629 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:51.378774 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:51.379411 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:51.379419 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:51.379619 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 06:55:51.379768 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 06:55:51.380142 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":511,"threads_started":1}
I20260812 06:55:51.386126 30587 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:37809
I20260812 06:55:51.386145 30804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:37809 every 8 connection(s)
I20260812 06:55:51.387544 30587 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:51.389761 30806 heartbeater.cc:347] Connected to a master server at 127.26.226.62:45859
I20260812 06:55:51.389999 30806 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:51.390635 30806 heartbeater.cc:511] Master 127.26.226.62:45859 requested a full tablet report, sending...
I20260812 06:55:51.392697 30469 ts_manager.cc:194] Registered new tserver with Master: efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809)
I20260812 06:55:51.394102 30469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:55295
I20260812 06:55:51.397444 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 30587
I20260812 06:55:51.397607 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:51.398211 27528 external_mini_cluster.cc:1428] Setting key cdb93fdb433927132cf3864a4f25d8e0
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:55:51.446842 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:45859
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:51.794265 30812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:51.794549 30812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:51.794656 30812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:51.802098 30812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:51.802222 30812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:51.802311 30812 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 06:55:51.802366 30812 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:55:51.802429 30812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:55:51.812394 30812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.226.62:45859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:51.813671 30812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:51.814977 30812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:51.828830 30818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:51.828732 30825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:51.828833 30822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:51.829927 30812 server_base.cc:1034] running on GCE node
I20260812 06:55:51.830660 30812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:51.831702 30812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:51.832973 30812 hybrid_clock.cc:648] HybridClock initialized: now 1786517751832904 us; error 42 us; skew 500 ppm
I20260812 06:55:51.833340 30812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:51 dist-test-slave-t3q3 krb5kdc[30392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517751, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:51.836522 30812 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:55:51.838054 30812 webserver.cc:460] Webserver started at http://127.26.226.2:41045/ using document root <none> and password file <none>
I20260812 06:55:51.838761 30812 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:51.838915 30812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:51.839262 30812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:51.842301 30812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "07a691a92dae43d180c025475bed6a8b"
format_stamp: "Formatted at 2026-08-12 06:55:51 on dist-test-slave-t3q3"
server_key: "b50fd534d902b30f4636c584ecccb6e3"
server_key_iv: "88e59309ed1cd8648afd618e0d8c6d76"
server_key_version: "encryptionkey@0"
I20260812 06:55:51.843119 30812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "07a691a92dae43d180c025475bed6a8b"
format_stamp: "Formatted at 2026-08-12 06:55:51 on dist-test-slave-t3q3"
server_key: "b50fd534d902b30f4636c584ecccb6e3"
server_key_iv: "88e59309ed1cd8648afd618e0d8c6d76"
server_key_version: "encryptionkey@0"
I20260812 06:55:51.848431 30812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 06:55:51.852123 30839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:51.853477 30812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:55:51.853657 30812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "07a691a92dae43d180c025475bed6a8b"
format_stamp: "Formatted at 2026-08-12 06:55:51 on dist-test-slave-t3q3"
server_key: "b50fd534d902b30f4636c584ecccb6e3"
server_key_iv: "88e59309ed1cd8648afd618e0d8c6d76"
server_key_version: "encryptionkey@0"
I20260812 06:55:51.853873 30812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:51.891389 30812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:51.895921 30812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:51.896281 30812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:51.899145 30812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:51.899312 30812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:51.899461 30812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:51.899560 30812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:55:51 dist-test-slave-t3q3 krb5kdc[30392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517751, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:51.928947 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.899811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:37463 (local address 127.26.226.62:45859)
0812 06:55:51.900597 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:51.900606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:51.901809 (+ 1203us) server_negotiation.cc:407] Connection header received
0812 06:55:51.905048 (+ 3239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:51.905056 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:51.905159 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:51.905313 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:51.907864 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.909175 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.912670 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.913483 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.913642 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.919642 (+ 6000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:51.919648 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:51.919651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:51.919689 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:51.923792 (+ 4103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:51.925074 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:51.925081 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:51.925087 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:51.925285 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:51.925841 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:51.925847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:51.925850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:51.926140 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:51.926329 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:51.928295 (+ 1966us) server_negotiation.cc:299] Negotiation successful
0812 06:55:51.928548 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":516,"threads_started":1}
I20260812 06:55:51.929584 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.900661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:45859 (local address 127.26.226.2:37463)
0812 06:55:51.901586 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:51.901653 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:51.904613 (+ 2960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:51.905630 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:51.905645 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:51.906550 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:51.907530 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.907565 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.909419 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.909427 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.912469 (+ 3042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.912495 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.913819 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.913827 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.914265 (+ 438us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.915238 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:51.915286 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:51.919378 (+ 4092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:51.923974 (+ 4596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:51.923991 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:51.924012 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:51.924904 (+ 892us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:51.925426 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:51.925434 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:51.925443 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:51.925689 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:51.926301 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:51.926309 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:51.926868 (+ 559us) client_negotiation.cc:769] Sending connection context
0812 06:55:51.927284 (+ 416us) client_negotiation.cc:240] Negotiation successful
0812 06:55:51.927948 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":365,"threads_started":1}
I20260812 06:55:51.943851 30812 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:42161
I20260812 06:55:51.943888 31026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:42161 every 8 connection(s)
I20260812 06:55:51.945363 30812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:51.947840 31027 heartbeater.cc:347] Connected to a master server at 127.26.226.62:45859
I20260812 06:55:51.948124 31027 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:51.949059 31027 heartbeater.cc:511] Master 127.26.226.62:45859 requested a full tablet report, sending...
I20260812 06:55:51.949448 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 30812
I20260812 06:55:51.949648 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:51.950246 27528 external_mini_cluster.cc:1428] Setting key 9f25ff1ef32899256c1cefaec6e69cc9
I20260812 06:55:51.950806 30469 ts_manager.cc:194] Registered new tserver with Master: 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161)
I20260812 06:55:51.952013 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.936528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:55147 (local address 127.26.226.62:45859)
0812 06:55:51.937092 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:51.937100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:51.937683 (+ 583us) server_negotiation.cc:407] Connection header received
0812 06:55:51.937917 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:51.937926 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:51.938042 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:51.938193 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:51.939377 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.940493 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.942201 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.942642 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.942746 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.945089 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:51.945095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:51.945099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:51.945140 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:51.948729 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:51.949519 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:51.949570 (+ 51us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:51.949577 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:51.949761 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:51.950214 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:51.950220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:51.950223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:51.950716 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:51.950928 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:51.951388 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 06:55:51.951654 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403}
I20260812 06:55:51.952061 31002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.936671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:45859 (local address 127.26.226.2:55147)
0812 06:55:51.937544 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:51.937571 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:51.937758 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:51.938204 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:51.938207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:51.938684 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:51.939217 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.939230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.940647 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.940658 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.941954 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.941992 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.942872 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.942883 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.943161 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.944008 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:51.944038 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:51.944877 (+ 839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:51.948860 (+ 3983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:51.948864 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:51.948867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:51.949391 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:51.949871 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:51.949877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:51.949880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:51.950061 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:51.950902 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:51.950908 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:51.951115 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 06:55:51.951281 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 06:55:51.951615 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":527,"threads_started":1}
I20260812 06:55:51.952268 30469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:37463
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:55:51.999605 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:45859
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:52.345468 31034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:52.345749 31034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:52.345966 31034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:52.353250 31034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:52.353389 31034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:52.353559 31034 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 06:55:52.353644 31034 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:55:52.353776 31034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:55:52.364008 31034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.226.62:45859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:52.365388 31034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:52.366743 31034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:52.380270 31045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:52.380362 31041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:52.380303 31042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:52.381469 31034 server_base.cc:1034] running on GCE node
I20260812 06:55:52.382032 31034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:52.383041 31034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:52.384246 31034 hybrid_clock.cc:648] HybridClock initialized: now 1786517752384209 us; error 34 us; skew 500 ppm
I20260812 06:55:52.384565 31034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:52 dist-test-slave-t3q3 krb5kdc[30392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517752, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:52.388692 31034 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:55:52.390305 31034 webserver.cc:460] Webserver started at http://127.26.226.3:42633/ using document root <none> and password file <none>
I20260812 06:55:52.390988 31034 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:52.391125 31034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:52.391520 31034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:52.394554 31034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "348279277cca4a5ba93f85635ed27ff7"
format_stamp: "Formatted at 2026-08-12 06:55:52 on dist-test-slave-t3q3"
server_key: "20a886eeab5fc9f4317294a7cc167966"
server_key_iv: "24ff830b53a60a93d4daddbf18abec75"
server_key_version: "encryptionkey@0"
I20260812 06:55:52.395346 31034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "348279277cca4a5ba93f85635ed27ff7"
format_stamp: "Formatted at 2026-08-12 06:55:52 on dist-test-slave-t3q3"
server_key: "20a886eeab5fc9f4317294a7cc167966"
server_key_iv: "24ff830b53a60a93d4daddbf18abec75"
server_key_version: "encryptionkey@0"
I20260812 06:55:52.396677 30806 heartbeater.cc:503] Master 127.26.226.62:45859 was elected leader, sending a full tablet report...
I20260812 06:55:52.401346 31034 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260812 06:55:52.404837 31056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:52.406189 31034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260812 06:55:52.406375 31034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "348279277cca4a5ba93f85635ed27ff7"
format_stamp: "Formatted at 2026-08-12 06:55:52 on dist-test-slave-t3q3"
server_key: "20a886eeab5fc9f4317294a7cc167966"
server_key_iv: "24ff830b53a60a93d4daddbf18abec75"
server_key_version: "encryptionkey@0"
I20260812 06:55:52.406594 31034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:52.434937 31034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:52.439296 31034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:52.439642 31034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:52.442520 31034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:52.442675 31034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:52.442819 31034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:52.442922 31034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:55:52 dist-test-slave-t3q3 krb5kdc[30392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517752, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:52.470410 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.443303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:54343 (local address 127.26.226.62:45859)
0812 06:55:52.443510 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.443517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.445000 (+ 1483us) server_negotiation.cc:407] Connection header received
0812 06:55:52.448247 (+ 3247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.448253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.448358 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.448473 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:52.450874 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.452345 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.455819 (+ 3474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.456633 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.456811 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.462705 (+ 5894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:52.462715 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:52.462722 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:52.462779 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:52.466496 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:52.467635 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:52.467644 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:52.467650 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:52.467908 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:52.468493 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:52.468499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:52.468503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:52.468827 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:52.468990 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.469782 (+ 792us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.470016 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 06:55:52.471735 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.443820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:45859 (local address 127.26.226.3:54343)
0812 06:55:52.444799 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.444857 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.447803 (+ 2946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.448830 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.448845 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.449747 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:52.450631 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.450667 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.452615 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.452624 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.455602 (+ 2978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.455629 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.456960 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.456971 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.457376 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.458506 (+ 1130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:52.458542 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:52.462403 (+ 3861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:52.466678 (+ 4275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:52.466691 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:52.466710 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:52.467469 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:52.468106 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:52.468112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:52.468116 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:52.468342 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:52.468987 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:52.468997 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:52.469386 (+ 389us) client_negotiation.cc:769] Sending connection context
0812 06:55:52.469666 (+ 280us) client_negotiation.cc:240] Negotiation successful
0812 06:55:52.470253 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":416,"threads_started":1}
I20260812 06:55:52.495636 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.478690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:35147 (local address 127.26.226.62:45859)
0812 06:55:52.478893 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.478901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.479671 (+ 770us) server_negotiation.cc:407] Connection header received
0812 06:55:52.479886 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.479890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.479951 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.480034 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:52.481841 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.482922 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.485693 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.486247 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.486399 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.488357 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:52.488363 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:52.488370 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:52.488409 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:52.492357 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:52.493362 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:52.493368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:52.493371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:52.493562 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:52.494092 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:52.494098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:52.494103 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:52.494412 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:52.494585 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.495000 (+ 415us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.495237 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:55:52.495787 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.478715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:45859 (local address 127.26.226.3:35147)
0812 06:55:52.479535 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.479569 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.479757 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.480131 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.480139 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.481008 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:52.481618 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.481642 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.483156 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.483164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.485523 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.485538 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.486470 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.486479 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.486643 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.487307 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:52.487325 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:52.488162 (+ 837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:52.492498 (+ 4336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:52.492507 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:52.492512 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:52.493224 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:52.493690 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:52.493696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:52.493706 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:52.493923 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:52.494543 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:52.494548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:52.494748 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 06:55:52.494891 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 06:55:52.495266 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":415,"threads_started":1}
I20260812 06:55:52.501075 31034 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:42839
I20260812 06:55:52.501116 31257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:42839 every 8 connection(s)
I20260812 06:55:52.502339 31034 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:52.503400 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 31034
I20260812 06:55:52.503597 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:52.504174 27528 external_mini_cluster.cc:1428] Setting key 0a82acc48175e3de1b58be8de63c534c
I20260812 06:55:52.504681 31258 heartbeater.cc:347] Connected to a master server at 127.26.226.62:45859
I20260812 06:55:52.504916 31258 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:52.505612 31258 heartbeater.cc:511] Master 127.26.226.62:45859 requested a full tablet report, sending...
I20260812 06:55:52.507149 30468 ts_manager.cc:194] Registered new tserver with Master: 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839)
I20260812 06:55:52.508224 30468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:54343
I20260812 06:55:52.519383 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:55:52 dist-test-slave-t3q3 krb5kdc[30392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:55:52 dist-test-slave-t3q3 krb5kdc[30392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:52.573525 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.558909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41382 (local address 127.26.226.62:45859)
0812 06:55:52.559154 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.559163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.559469 (+ 306us) server_negotiation.cc:407] Connection header received
0812 06:55:52.559693 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.559704 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.559772 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.559884 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:52.561435 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.562215 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.563520 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.563964 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.564140 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.568173 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:52.568178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:52.568181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:52.568223 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:52.570657 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:52.571344 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:52.571350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:52.571354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:52.571489 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:52.571896 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:52.571901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:52.571910 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:52.572270 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:52.572582 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.572898 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.573165 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 06:55:52.577894 30468 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:52.586603 30468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:52.610572 31284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.599302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.26.226.1:37809)
0812 06:55:52.599968 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.599978 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.600739 (+ 761us) server_negotiation.cc:407] Connection header received
0812 06:55:52.602817 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.602829 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.603080 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.603306 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:52.603857 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.605411 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.607784 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.609341 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.609480 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.609689 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.610062 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.610261 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":316,"threads_started":1}
I20260812 06:55:52.610730 31283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.599103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:37809 (local address 127.0.0.1:51016)
0812 06:55:52.600527 (+ 1424us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.600577 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.600953 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.603296 (+ 2343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.603301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.603326 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:52.603686 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.603696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.605574 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.605580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.607582 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.607596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.609536 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.609543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.609792 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.609850 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 06:55:52.609994 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 06:55:52.610219 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1105,"mutex_wait_us":45,"thread_start_us":2230,"threads_started":1}
I20260812 06:55:52.611364 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.599195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:42839 (local address 127.0.0.1:57750)
0812 06:55:52.601633 (+ 2438us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.601650 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.601788 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.602550 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.602559 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.602591 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:52.603006 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.603022 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.604984 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.604988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.607530 (+ 2542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.607548 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.609807 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.609813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.609993 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.610022 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 06:55:52.610101 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 06:55:52.610219 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2302,"mutex_wait_us":83,"thread_start_us":1344,"threads_started":1}
I20260812 06:55:52.611557 31285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.599881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.26.226.3:42839)
0812 06:55:52.600698 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.600707 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.601791 (+ 1084us) server_negotiation.cc:407] Connection header received
0812 06:55:52.602009 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.602019 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.602306 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.602501 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:52.603284 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.604819 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.607739 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.609232 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.609366 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.609631 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.610390 (+ 759us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.610565 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":1626,"threads_started":1}
I20260812 06:55:52.612113 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.601315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:42161 (local address 127.0.0.1:40982)
0812 06:55:52.602252 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.602270 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.602379 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.604167 (+ 1788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.604172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.604209 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:52.604560 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.604572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.606382 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.606388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.609110 (+ 2722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.609128 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.611194 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.611206 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.611419 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.611451 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:55:52.611549 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 06:55:52.611678 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":4,"thread_start_us":1234,"threads_started":1}
I20260812 06:55:52.612139 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.601564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40982 (local address 127.26.226.2:42161)
0812 06:55:52.603255 (+ 1691us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.603263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.603287 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:55:52.603388 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.603398 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.603622 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.603838 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:52.604767 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.606213 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.609323 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.610668 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.610823 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.611018 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.611607 (+ 589us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.611788 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1515,"thread_start_us":1386,"threads_started":1}
I20260812 06:55:52.615115 31135 tablet_service.cc:1467] Processing CreateTablet for tablet cb5477f8b5e24533b9ca69686fba0f4d (DEFAULT_TABLE table=test-table [id=7567b92cd05d416b8f83ca2f6ea2b51f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:52.615228 30697 tablet_service.cc:1467] Processing CreateTablet for tablet cb5477f8b5e24533b9ca69686fba0f4d (DEFAULT_TABLE table=test-table [id=7567b92cd05d416b8f83ca2f6ea2b51f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:52.615747 30920 tablet_service.cc:1467] Processing CreateTablet for tablet cb5477f8b5e24533b9ca69686fba0f4d (DEFAULT_TABLE table=test-table [id=7567b92cd05d416b8f83ca2f6ea2b51f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:52.616577 31135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5477f8b5e24533b9ca69686fba0f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:52.616722 30697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5477f8b5e24533b9ca69686fba0f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:52.617069 30920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5477f8b5e24533b9ca69686fba0f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:52.628328 31290 tablet_bootstrap.cc:492] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: Bootstrap starting.
I20260812 06:55:52.631515 31292 tablet_bootstrap.cc:492] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: Bootstrap starting.
I20260812 06:55:52.632757 31290 tablet_bootstrap.cc:654] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:52.632783 31294 tablet_bootstrap.cc:492] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: Bootstrap starting.
I20260812 06:55:52.634256 31290 log.cc:824] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:52.635656 31292 tablet_bootstrap.cc:654] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:52.636029 31294 tablet_bootstrap.cc:654] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:52.637019 31292 log.cc:824] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:52.637055 31294 log.cc:824] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:52.637820 31290 tablet_bootstrap.cc:492] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: No bootstrap required, opened a new log
I20260812 06:55:52.638198 31290 ts_tablet_manager.cc:1397] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 06:55:52.639652 31294 tablet_bootstrap.cc:492] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: No bootstrap required, opened a new log
I20260812 06:55:52.639940 31294 ts_tablet_manager.cc:1397] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:55:52.647032 31292 tablet_bootstrap.cc:492] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: No bootstrap required, opened a new log
I20260812 06:55:52.647344 31292 ts_tablet_manager.cc:1397] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260812 06:55:52.647866 31294 raft_consensus.cc:348] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } }
I20260812 06:55:52.648337 31294 raft_consensus.cc:374] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:52.648453 31294 raft_consensus.cc:729] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a691a92dae43d180c025475bed6a8b, State: Initialized, Role: FOLLOWER
I20260812 06:55:52.649119 31294 consensus_queue.cc:260] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } }
I20260812 06:55:52.650288 31027 heartbeater.cc:503] Master 127.26.226.62:45859 was elected leader, sending a full tablet report...
I20260812 06:55:52.650352 31290 raft_consensus.cc:348] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } }
I20260812 06:55:52.650823 31290 raft_consensus.cc:374] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:52.650935 31294 ts_tablet_manager.cc:1428] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260812 06:55:52.650940 31290 raft_consensus.cc:729] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348279277cca4a5ba93f85635ed27ff7, State: Initialized, Role: FOLLOWER
I20260812 06:55:52.651558 31290 consensus_queue.cc:260] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } }
I20260812 06:55:52.660810 31258 heartbeater.cc:503] Master 127.26.226.62:45859 was elected leader, sending a full tablet report...
I20260812 06:55:52.661680 31290 ts_tablet_manager.cc:1428] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: Time spent starting tablet: real 0.023s user 0.009s sys 0.013s
I20260812 06:55:52.663293 31292 raft_consensus.cc:348] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } }
I20260812 06:55:52.663851 31292 raft_consensus.cc:374] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:52.663998 31292 raft_consensus.cc:729] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd7cbf9aeb0417eb8f3895be54521a6, State: Initialized, Role: FOLLOWER
I20260812 06:55:52.664781 31292 consensus_queue.cc:260] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } }
I20260812 06:55:52.666626 31292 ts_tablet_manager.cc:1428] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: Time spent starting tablet: real 0.019s user 0.014s sys 0.004s
W20260812 06:55:52.697772 31028 tablet.cc:2369] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:55:52.754276 31259 tablet.cc:2369] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:55:52.808388 31306 raft_consensus.cc:482] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:52.808769 31306 raft_consensus.cc:504] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } }
I20260812 06:55:52.810461 31306 leader_election.cc:288] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839), 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161)
I20260812 06:55:52.817524 31285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.811032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:46151 (local address 127.26.226.3:42839)
0812 06:55:52.811287 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.811298 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.811828 (+ 530us) server_negotiation.cc:407] Connection header received
0812 06:55:52.812079 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.812085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.812192 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.812294 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:52.812977 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.813696 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.815576 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.816576 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.816784 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.816860 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.817127 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.817254 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:55:52.817679 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.810928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:42839 (local address 127.26.226.1:46151)
0812 06:55:52.811662 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.811684 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.811880 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.812389 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.812394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.812425 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:52.812813 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.812822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.813832 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.813840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.815413 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.815426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.816789 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.816795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.816955 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.816985 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:55:52.817101 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 06:55:52.817245 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":53,"thread_start_us":427,"threads_started":1}
I20260812 06:55:52.817768 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.812023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:46765 (local address 127.26.226.2:42161)
0812 06:55:52.812339 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.812345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.812606 (+ 261us) server_negotiation.cc:407] Connection header received
0812 06:55:52.812843 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.812846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.812901 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.813015 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:52.813534 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.814251 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.815937 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.816822 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.817011 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.817087 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.817363 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.817487 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260812 06:55:52.817991 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.811845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:42161 (local address 127.26.226.1:46765)
0812 06:55:52.812493 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.812511 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.812687 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.813031 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.813035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.813055 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:52.813376 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.813384 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.814391 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.814402 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.815765 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.815776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.817041 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.817048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.817204 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.817228 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:55:52.817335 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 06:55:52.817467 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":1,"thread_start_us":316,"threads_started":1}
I20260812 06:55:52.818910 31167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5477f8b5e24533b9ca69686fba0f4d" candidate_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348279277cca4a5ba93f85635ed27ff7" is_pre_election: true
I20260812 06:55:52.818936 30951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5477f8b5e24533b9ca69686fba0f4d" candidate_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a691a92dae43d180c025475bed6a8b" is_pre_election: true
I20260812 06:55:52.819471 30951 raft_consensus.cc:2454] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd7cbf9aeb0417eb8f3895be54521a6 in term 0.
I20260812 06:55:52.819464 31167 raft_consensus.cc:2454] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd7cbf9aeb0417eb8f3895be54521a6 in term 0.
I20260812 06:55:52.820248 30609 leader_election.cc:302] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a691a92dae43d180c025475bed6a8b, efd7cbf9aeb0417eb8f3895be54521a6; no voters:
I20260812 06:55:52.820681 31306 raft_consensus.cc:2781] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:52.820855 31306 raft_consensus.cc:482] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:52.820974 31306 raft_consensus.cc:3037] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:52.823294 31306 raft_consensus.cc:504] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } }
I20260812 06:55:52.823945 31306 leader_election.cc:288] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [CANDIDATE]: Term 1 election: Requested vote from peers 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839), 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161)
I20260812 06:55:52.824669 31167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5477f8b5e24533b9ca69686fba0f4d" candidate_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348279277cca4a5ba93f85635ed27ff7"
I20260812 06:55:52.824669 30951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5477f8b5e24533b9ca69686fba0f4d" candidate_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a691a92dae43d180c025475bed6a8b"
I20260812 06:55:52.824916 31167 raft_consensus.cc:3037] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:52.824918 30951 raft_consensus.cc:3037] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:52.828325 31167 raft_consensus.cc:2454] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd7cbf9aeb0417eb8f3895be54521a6 in term 1.
I20260812 06:55:52.828325 30951 raft_consensus.cc:2454] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd7cbf9aeb0417eb8f3895be54521a6 in term 1.
I20260812 06:55:52.828823 30609 leader_election.cc:302] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a691a92dae43d180c025475bed6a8b, efd7cbf9aeb0417eb8f3895be54521a6; no voters:
I20260812 06:55:52.829109 31306 raft_consensus.cc:2781] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:52.829598 31306 raft_consensus.cc:686] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 LEADER]: Becoming Leader. State: Replica: efd7cbf9aeb0417eb8f3895be54521a6, State: Running, Role: LEADER
I20260812 06:55:52.830116 31306 consensus_queue.cc:237] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } }
I20260812 06:55:52.835283 30468 catalog_manager.cc:5495] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1). New cstate: current_term: 1 leader_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } health_report { overall_health: UNKNOWN } } }
Aug 12 06:55:52 dist-test-slave-t3q3 krb5kdc[30392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517750, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
W20260812 06:55:52.890344 30807 tablet.cc:2369] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:55:52.892839 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.878849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41384 (local address 127.26.226.62:45859)
0812 06:55:52.879063 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.879071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.879652 (+ 581us) server_negotiation.cc:407] Connection header received
0812 06:55:52.879815 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.879819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.879898 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.879983 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:52.881184 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.881880 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.883177 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.883585 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.883752 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.887231 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:52.887235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:52.887238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:52.887267 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:52.889978 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:52.890829 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:52.890835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:52.890838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:52.890991 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:52.891363 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:52.891366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:52.891368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:52.891637 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:52.891777 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.892285 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.892574 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:55:52.893020 31321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.878849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:45859 (local address 127.0.0.1:41384)
0812 06:55:52.879547 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.879564 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.879697 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.880029 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.880032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.880772 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:52.881050 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.881059 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.882001 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.882004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.883024 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.883036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.883810 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.883822 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.884002 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.884690 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:52.884709 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:52.887039 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:52.890160 (+ 3121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:52.890172 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:52.890179 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:52.890708 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:52.891116 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:52.891119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:52.891122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:52.891258 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:52.891768 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:52.891774 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:52.891923 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 06:55:52.892114 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 06:55:52.892426 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":418,"threads_started":1}
I20260812 06:55:52.898144 30468 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:41384:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 06:55:52.899050 30468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:52.905827 30920 tablet_service.cc:1467] Processing CreateTablet for tablet 7e35f5a4917c489b80c8627eb8e5f833 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2b2b8c7f54174b0499b010bc4c4f6883]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:55:52.905764 30697 tablet_service.cc:1467] Processing CreateTablet for tablet 7e35f5a4917c489b80c8627eb8e5f833 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2b2b8c7f54174b0499b010bc4c4f6883]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:55:52.906088 31135 tablet_service.cc:1467] Processing CreateTablet for tablet 7e35f5a4917c489b80c8627eb8e5f833 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2b2b8c7f54174b0499b010bc4c4f6883]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:55:52.906828 30920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e35f5a4917c489b80c8627eb8e5f833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:52.906829 30697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e35f5a4917c489b80c8627eb8e5f833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:52.906925 31135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e35f5a4917c489b80c8627eb8e5f833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:52.913318 31290 tablet_bootstrap.cc:492] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: Bootstrap starting.
I20260812 06:55:52.914232 31292 tablet_bootstrap.cc:492] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: Bootstrap starting.
I20260812 06:55:52.914271 31294 tablet_bootstrap.cc:492] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: Bootstrap starting.
I20260812 06:55:52.914989 31290 tablet_bootstrap.cc:654] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:52.915946 31292 tablet_bootstrap.cc:654] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:52.916447 31294 tablet_bootstrap.cc:654] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:52.917760 31290 tablet_bootstrap.cc:492] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: No bootstrap required, opened a new log
I20260812 06:55:52.917949 31290 ts_tablet_manager.cc:1397] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 06:55:52.918347 31292 tablet_bootstrap.cc:492] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: No bootstrap required, opened a new log
I20260812 06:55:52.918510 31292 ts_tablet_manager.cc:1397] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 06:55:52.918725 31290 raft_consensus.cc:348] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:52.919008 31290 raft_consensus.cc:374] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:52.919099 31294 tablet_bootstrap.cc:492] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: No bootstrap required, opened a new log
I20260812 06:55:52.919117 31290 raft_consensus.cc:729] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348279277cca4a5ba93f85635ed27ff7, State: Initialized, Role: FOLLOWER
I20260812 06:55:52.919248 31294 ts_tablet_manager.cc:1397] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 06:55:52.919286 31292 raft_consensus.cc:348] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:52.919377 31290 consensus_queue.cc:260] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:52.919530 31292 raft_consensus.cc:374] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:52.919988 31290 ts_tablet_manager.cc:1428] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 06:55:52.920025 31294 raft_consensus.cc:348] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:52.919914 31305 tablet_replica.cc:439] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } }
I20260812 06:55:52.920282 31294 raft_consensus.cc:374] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:52.920346 31305 tablet_replica.cc:442] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:52.920380 31294 raft_consensus.cc:729] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a691a92dae43d180c025475bed6a8b, State: Initialized, Role: FOLLOWER
I20260812 06:55:52.920636 31294 consensus_queue.cc:260] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:52.921208 31292 raft_consensus.cc:729] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd7cbf9aeb0417eb8f3895be54521a6, State: Initialized, Role: FOLLOWER
I20260812 06:55:52.921106 31304 tablet_replica.cc:439] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } }
I20260812 06:55:52.921495 31304 tablet_replica.cc:442] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:52.921510 31292 consensus_queue.cc:260] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:52.921984 31306 tablet_replica.cc:439] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } }
I20260812 06:55:52.922287 31306 tablet_replica.cc:442] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:52.923102 31294 ts_tablet_manager.cc:1428] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260812 06:55:52.923175 31292 ts_tablet_manager.cc:1428] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260812 06:55:53.139647 31305 raft_consensus.cc:482] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:53.139897 31305 raft_consensus.cc:504] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.141605 31305 leader_election.cc:288] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809), 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161)
I20260812 06:55:53.151217 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.142028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:40317 (local address 127.26.226.1:37809)
0812 06:55:53.144191 (+ 2163us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.144196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.144219 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:55:53.144278 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.144281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.144352 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.144440 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:53.144979 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.146063 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.148710 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.149932 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.150113 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.150191 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.150777 (+ 586us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.150921 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2002,"thread_start_us":1859,"threads_started":1}
I20260812 06:55:53.151219 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.142531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:44127 (local address 127.26.226.2:42161)
0812 06:55:53.142811 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.142821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.143786 (+ 965us) server_negotiation.cc:407] Connection header received
0812 06:55:53.144039 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.144044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.144167 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.144266 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:53.144740 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.145935 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.148699 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.149923 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.150108 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.150191 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.150777 (+ 586us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.150915 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260812 06:55:53.151367 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.141812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:37809 (local address 127.26.226.3:40317)
0812 06:55:53.143470 (+ 1658us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.143496 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.143644 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.144463 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.144472 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.144497 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:53.144812 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.144822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.146190 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.146195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.148508 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.148523 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.150209 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.150214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.150484 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.150624 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.150730 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.150907 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1528,"mutex_wait_us":203,"thread_start_us":1699,"threads_started":1}
I20260812 06:55:53.151753 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.141957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:42161 (local address 127.26.226.3:44127)
0812 06:55:53.143691 (+ 1734us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.143709 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.143880 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.144291 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.144296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.144322 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:53.144549 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.144565 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.146106 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.146111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.148507 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.148523 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.150171 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.150180 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.150481 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.150627 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.150730 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.150908 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1649,"mutex_wait_us":266,"thread_start_us":1279,"threads_started":1}
I20260812 06:55:53.152330 30738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e35f5a4917c489b80c8627eb8e5f833" candidate_uuid: "348279277cca4a5ba93f85635ed27ff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" is_pre_election: true
I20260812 06:55:53.152500 30951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e35f5a4917c489b80c8627eb8e5f833" candidate_uuid: "348279277cca4a5ba93f85635ed27ff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a691a92dae43d180c025475bed6a8b" is_pre_election: true
I20260812 06:55:53.152760 30738 raft_consensus.cc:2454] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 348279277cca4a5ba93f85635ed27ff7 in term 0.
I20260812 06:55:53.152802 30951 raft_consensus.cc:2454] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 348279277cca4a5ba93f85635ed27ff7 in term 0.
I20260812 06:55:53.153345 31062 leader_election.cc:302] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 348279277cca4a5ba93f85635ed27ff7, efd7cbf9aeb0417eb8f3895be54521a6; no voters:
I20260812 06:55:53.153776 31305 raft_consensus.cc:2781] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:53.153964 31305 raft_consensus.cc:482] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:53.154095 31305 raft_consensus.cc:3037] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:53.156718 31305 raft_consensus.cc:504] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.157371 31305 leader_election.cc:288] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [CANDIDATE]: Term 1 election: Requested vote from peers efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809), 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161)
I20260812 06:55:53.157917 30951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e35f5a4917c489b80c8627eb8e5f833" candidate_uuid: "348279277cca4a5ba93f85635ed27ff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a691a92dae43d180c025475bed6a8b"
I20260812 06:55:53.157920 30738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e35f5a4917c489b80c8627eb8e5f833" candidate_uuid: "348279277cca4a5ba93f85635ed27ff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd7cbf9aeb0417eb8f3895be54521a6"
I20260812 06:55:53.158151 30951 raft_consensus.cc:3037] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:53.158156 30738 raft_consensus.cc:3037] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:53.160298 30951 raft_consensus.cc:2454] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 348279277cca4a5ba93f85635ed27ff7 in term 1.
I20260812 06:55:53.160516 30738 raft_consensus.cc:2454] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 348279277cca4a5ba93f85635ed27ff7 in term 1.
I20260812 06:55:53.160779 31061 leader_election.cc:302] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a691a92dae43d180c025475bed6a8b, 348279277cca4a5ba93f85635ed27ff7; no voters:
I20260812 06:55:53.161128 31305 raft_consensus.cc:2781] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:53.161518 31305 raft_consensus.cc:686] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [term 1 LEADER]: Becoming Leader. State: Replica: 348279277cca4a5ba93f85635ed27ff7, State: Running, Role: LEADER
I20260812 06:55:53.162034 31305 consensus_queue.cc:237] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.165302 31328 tablet_replica.cc:439] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: TxnStatusTablet state changed. Reason: New leader 348279277cca4a5ba93f85635ed27ff7. Latest consensus state: current_term: 1 leader_uuid: "348279277cca4a5ba93f85635ed27ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } }
I20260812 06:55:53.165705 31328 tablet_replica.cc:442] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:55:53.166425 31335 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 06:55:53.166740 31335 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 06:55:53.166750 30468 catalog_manager.cc:5495] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3). New cstate: current_term: 1 leader_uuid: "348279277cca4a5ba93f85635ed27ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } health_report { overall_health: HEALTHY } } }
I20260812 06:55:53.216601 31321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.209699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:42839 (local address 127.0.0.1:57754)
0812 06:55:53.210013 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.210034 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.210213 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.210778 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.210782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.210793 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:53.211087 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.211098 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.212672 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.212676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.213776 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.213790 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.214754 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.214759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.214919 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.214992 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.216045 (+ 1053us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.216112 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.216201 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.216326 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 06:55:53.216674 31285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.209842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57754 (local address 127.26.226.3:42839)
0812 06:55:53.210073 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.210080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.210113 (+ 33us) server_negotiation.cc:407] Connection header received
0812 06:55:53.210449 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.210456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.210577 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.210752 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:53.211261 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.212536 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.213936 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.214586 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.214724 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.215189 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.215931 (+ 742us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.216117 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.216268 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.216354 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 06:55:53.224871 30738 raft_consensus.cc:1261] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Refusing update from remote peer 348279277cca4a5ba93f85635ed27ff7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:53.225003 30951 raft_consensus.cc:1261] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Refusing update from remote peer 348279277cca4a5ba93f85635ed27ff7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:53.225983 31328 consensus_queue.cc:1035] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:53.226390 31305 consensus_queue.cc:1035] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:53.235061 31306 tablet_replica.cc:439] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: TxnStatusTablet state changed. Reason: New leader 348279277cca4a5ba93f85635ed27ff7. Latest consensus state: current_term: 1 leader_uuid: "348279277cca4a5ba93f85635ed27ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } }
I20260812 06:55:53.235441 31306 tablet_replica.cc:442] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:53.235813 31306 consensus_queue.cc:1035] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:53.243736 31304 tablet_replica.cc:439] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: TxnStatusTablet state changed. Reason: New leader 348279277cca4a5ba93f85635ed27ff7. Latest consensus state: current_term: 1 leader_uuid: "348279277cca4a5ba93f85635ed27ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } }
I20260812 06:55:53.244052 31304 tablet_replica.cc:442] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:53.246219 31341 mvcc.cc:205] Tried to move back new op lower bound from 7317576717198729216 to 7317576716955672576. Current Snapshot: MvccSnapshot[applied={T|T < 7317576717198729216}]
I20260812 06:55:53.255978 31306 tablet_replica.cc:439] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "348279277cca4a5ba93f85635ed27ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } }
I20260812 06:55:53.256426 31306 tablet_replica.cc:442] T 7e35f5a4917c489b80c8627eb8e5f833 P efd7cbf9aeb0417eb8f3895be54521a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:53.263082 31309 consensus_queue.cc:1035] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:53.271546 31328 tablet_replica.cc:439] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "348279277cca4a5ba93f85635ed27ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } }
I20260812 06:55:53.271847 31328 tablet_replica.cc:442] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:55:53.272178 31328 tablet_replica.cc:439] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "348279277cca4a5ba93f85635ed27ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } }
I20260812 06:55:53.272418 31328 tablet_replica.cc:442] T 7e35f5a4917c489b80c8627eb8e5f833 P 348279277cca4a5ba93f85635ed27ff7: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:55:53.279500 31304 tablet_replica.cc:439] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "348279277cca4a5ba93f85635ed27ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } } }
I20260812 06:55:53.279819 31304 tablet_replica.cc:442] T 7e35f5a4917c489b80c8627eb8e5f833 P 07a691a92dae43d180c025475bed6a8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:53.291452 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.283972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.26.226.1:37809)
0812 06:55:53.284193 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.284203 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.284907 (+ 704us) server_negotiation.cc:407] Connection header received
0812 06:55:53.285228 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.285240 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.285343 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.285585 (+ 242us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:53.286148 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.287020 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.289123 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.289548 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.289654 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.290185 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.290782 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.290898 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.291041 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.291115 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:55:53.309983 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.303245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:42839 (local address 127.26.226.1:51013)
0812 06:55:53.304203 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.304222 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.304360 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.304766 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.304769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.304781 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:53.305072 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.305080 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.306489 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.306493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.307734 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.307748 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.308850 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.308854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.308969 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.309028 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.309483 (+ 455us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.309492 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.309553 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.309670 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":563,"threads_started":1}
I20260812 06:55:53.310004 31285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.303268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:51013 (local address 127.26.226.3:42839)
0812 06:55:53.303646 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.303655 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.304312 (+ 657us) server_negotiation.cc:407] Connection header received
0812 06:55:53.304515 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.304521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.304619 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.304724 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:53.305225 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.306362 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.307885 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.308675 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.308816 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.309235 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.309378 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.309490 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.309603 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.309690 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260812 06:55:53.359376 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.352253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:42793 (local address 127.26.226.1:37809)
0812 06:55:53.352478 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.352488 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.353178 (+ 690us) server_negotiation.cc:407] Connection header received
0812 06:55:53.353396 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.353403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.353496 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.353614 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:53.354408 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.355477 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.357364 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.357799 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.357937 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.358548 (+ 611us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.358696 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.358797 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.358957 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.359041 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:55:53.359797 31368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.352167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:37809 (local address 127.26.226.3:42793)
0812 06:55:53.353064 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.353085 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.353238 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.353635 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.353639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.353651 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:53.354145 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.354158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.355615 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.355623 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.357176 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.357196 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.358007 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.358014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.358209 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.358284 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.358810 (+ 526us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.358821 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.358923 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.359068 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":501,"threads_started":1}
I20260812 06:55:53.395987 30469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:53.396332 30469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:53.400513 30469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb: Sending DeleteTablet for 3 replicas of tablet cb5477f8b5e24533b9ca69686fba0f4d
I20260812 06:55:53.401741 31135 tablet_service.cc:1514] Processing DeleteTablet for tablet cb5477f8b5e24533b9ca69686fba0f4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:53 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:57750
I20260812 06:55:53.401808 30697 tablet_service.cc:1514] Processing DeleteTablet for tablet cb5477f8b5e24533b9ca69686fba0f4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:53 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:51016
I20260812 06:55:53.403364 30920 tablet_service.cc:1514] Processing DeleteTablet for tablet cb5477f8b5e24533b9ca69686fba0f4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:53 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:40982
I20260812 06:55:53.405725 30468 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:53.406579 30468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:53.415001 30919 tablet_service.cc:1467] Processing CreateTablet for tablet ae7ff2e852e64157a53ff901d22378fb (DEFAULT_TABLE table=test-table [id=db1141b60db7485299d531af22547698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:53.415014 31134 tablet_service.cc:1467] Processing CreateTablet for tablet ae7ff2e852e64157a53ff901d22378fb (DEFAULT_TABLE table=test-table [id=db1141b60db7485299d531af22547698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:53.415405 30696 tablet_service.cc:1467] Processing CreateTablet for tablet ae7ff2e852e64157a53ff901d22378fb (DEFAULT_TABLE table=test-table [id=db1141b60db7485299d531af22547698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:53.415597 30919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae7ff2e852e64157a53ff901d22378fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:53.415597 31134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae7ff2e852e64157a53ff901d22378fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:53.415969 30696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae7ff2e852e64157a53ff901d22378fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:53.422516 31294 tablet_bootstrap.cc:492] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b: Bootstrap starting.
I20260812 06:55:53.424253 31294 tablet_bootstrap.cc:654] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:53.425336 31292 tablet_bootstrap.cc:492] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6: Bootstrap starting.
I20260812 06:55:53.426995 31292 tablet_bootstrap.cc:654] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:53.433779 31377 tablet_bootstrap.cc:492] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7: Bootstrap starting.
I20260812 06:55:53.435861 31377 tablet_bootstrap.cc:654] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:53.437644 31292 tablet_bootstrap.cc:492] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6: No bootstrap required, opened a new log
I20260812 06:55:53.437848 31292 ts_tablet_manager.cc:1397] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.001s
I20260812 06:55:53.438789 31292 raft_consensus.cc:348] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.439065 31292 raft_consensus.cc:374] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:53.439169 31292 raft_consensus.cc:729] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd7cbf9aeb0417eb8f3895be54521a6, State: Initialized, Role: FOLLOWER
I20260812 06:55:53.439447 31292 consensus_queue.cc:260] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.440204 31292 ts_tablet_manager.cc:1428] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 06:55:53.444658 31376 tablet_replica.cc:330] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: stopping tablet replica
I20260812 06:55:53.446389 31376 raft_consensus.cc:2229] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:53.446841 31376 raft_consensus.cc:2258] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:53.448989 31376 ts_tablet_manager.cc:1905] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:53.456331 31294 tablet_bootstrap.cc:492] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b: No bootstrap required, opened a new log
I20260812 06:55:53.456712 31294 ts_tablet_manager.cc:1397] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.005s
I20260812 06:55:53.457897 31294 raft_consensus.cc:348] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.458310 31294 raft_consensus.cc:374] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:53.458459 31294 raft_consensus.cc:729] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a691a92dae43d180c025475bed6a8b, State: Initialized, Role: FOLLOWER
I20260812 06:55:53.458977 31294 consensus_queue.cc:260] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.462265 31294 ts_tablet_manager.cc:1428] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260812 06:55:53.468753 31374 tablet_replica.cc:330] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: stopping tablet replica
I20260812 06:55:53.469463 31374 raft_consensus.cc:2229] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:53.470072 31374 raft_consensus.cc:2258] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:53.470923 31373 tablet_replica.cc:330] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: stopping tablet replica
I20260812 06:55:53.471455 31373 raft_consensus.cc:2229] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:53.472304 31373 raft_consensus.cc:2258] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:53.474186 31377 tablet_bootstrap.cc:492] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7: No bootstrap required, opened a new log
I20260812 06:55:53.474335 31377 ts_tablet_manager.cc:1397] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7: Time spent bootstrapping tablet: real 0.041s user 0.017s sys 0.000s
I20260812 06:55:53.475106 31377 raft_consensus.cc:348] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.475349 31377 raft_consensus.cc:374] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:53.475445 31377 raft_consensus.cc:729] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348279277cca4a5ba93f85635ed27ff7, State: Initialized, Role: FOLLOWER
I20260812 06:55:53.475656 31377 consensus_queue.cc:260] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.476410 31377 ts_tablet_manager.cc:1428] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:55:53.478401 31374 ts_tablet_manager.cc:1905] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:53.484356 31373 ts_tablet_manager.cc:1905] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:53.488083 31376 ts_tablet_manager.cc:1918] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:53.488289 31376 log.cc:1196] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/cb5477f8b5e24533b9ca69686fba0f4d
I20260812 06:55:53.488883 31376 ts_tablet_manager.cc:1939] T cb5477f8b5e24533b9ca69686fba0f4d P 07a691a92dae43d180c025475bed6a8b: Deleting consensus metadata
I20260812 06:55:53.490944 30443 catalog_manager.cc:4841] TS 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161): tablet cb5477f8b5e24533b9ca69686fba0f4d (table test-table [id=7567b92cd05d416b8f83ca2f6ea2b51f]) successfully deleted
I20260812 06:55:53.498471 31373 ts_tablet_manager.cc:1918] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:53.498767 31373 log.cc:1196] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/cb5477f8b5e24533b9ca69686fba0f4d
I20260812 06:55:53.499470 31373 ts_tablet_manager.cc:1939] T cb5477f8b5e24533b9ca69686fba0f4d P efd7cbf9aeb0417eb8f3895be54521a6: Deleting consensus metadata
I20260812 06:55:53.503049 30444 catalog_manager.cc:4841] TS efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809): tablet cb5477f8b5e24533b9ca69686fba0f4d (table test-table [id=7567b92cd05d416b8f83ca2f6ea2b51f]) successfully deleted
I20260812 06:55:53.505757 31374 ts_tablet_manager.cc:1918] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:53.505961 31374 log.cc:1196] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/cb5477f8b5e24533b9ca69686fba0f4d
I20260812 06:55:53.506613 31374 ts_tablet_manager.cc:1939] T cb5477f8b5e24533b9ca69686fba0f4d P 348279277cca4a5ba93f85635ed27ff7: Deleting consensus metadata
I20260812 06:55:53.508504 30443 catalog_manager.cc:4841] TS 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839): tablet cb5477f8b5e24533b9ca69686fba0f4d (table test-table [id=7567b92cd05d416b8f83ca2f6ea2b51f]) successfully deleted
I20260812 06:55:53.614601 31304 raft_consensus.cc:482] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:53.614989 31304 raft_consensus.cc:504] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.616808 31304 leader_election.cc:288] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809), 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839)
I20260812 06:55:53.636588 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.617814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:56189 (local address 127.26.226.1:37809)
0812 06:55:53.618091 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.618101 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.627977 (+ 9876us) server_negotiation.cc:407] Connection header received
0812 06:55:53.628668 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.628676 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.628780 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.628845 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:53.629696 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.630920 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.633793 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.635174 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.635341 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.635434 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.636108 (+ 674us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.636257 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 06:55:53.636610 31285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.618304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:34781 (local address 127.26.226.3:42839)
0812 06:55:53.618575 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.618583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.628745 (+ 10162us) server_negotiation.cc:407] Connection header received
0812 06:55:53.629151 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.629159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.629271 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.629337 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:53.630212 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.631317 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.633939 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.635376 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.635533 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.635619 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.636194 (+ 575us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.636335 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 06:55:53.637385 31387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.617279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:42839 (local address 127.26.226.2:34781)
0812 06:55:53.628650 (+ 11371us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.628669 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.628970 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.629555 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.629562 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.629591 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:53.629996 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.630009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.631539 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.631544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.633749 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.633765 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.635805 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.635812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.635977 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.636016 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.636895 (+ 879us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.637044 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11246,"thread_start_us":11466,"threads_started":1}
I20260812 06:55:53.637861 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.617004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:37809 (local address 127.26.226.2:56189)
0812 06:55:53.627802 (+ 10798us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.627831 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.628446 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.629064 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.629071 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.629124 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:53.629493 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.629507 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.631101 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.631106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.633558 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.633574 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.635640 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.635648 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.635869 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.635904 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.637194 (+ 1290us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.637321 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10570,"thread_start_us":11043,"threads_started":1}
I20260812 06:55:53.638144 31167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae7ff2e852e64157a53ff901d22378fb" candidate_uuid: "07a691a92dae43d180c025475bed6a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348279277cca4a5ba93f85635ed27ff7" is_pre_election: true
I20260812 06:55:53.638491 31167 raft_consensus.cc:2454] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a691a92dae43d180c025475bed6a8b in term 0.
I20260812 06:55:53.638645 30738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae7ff2e852e64157a53ff901d22378fb" candidate_uuid: "07a691a92dae43d180c025475bed6a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" is_pre_election: true
I20260812 06:55:53.638931 30738 raft_consensus.cc:2454] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a691a92dae43d180c025475bed6a8b in term 0.
I20260812 06:55:53.639112 30841 leader_election.cc:302] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a691a92dae43d180c025475bed6a8b, 348279277cca4a5ba93f85635ed27ff7; no voters:
I20260812 06:55:53.639618 31304 raft_consensus.cc:2781] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:53.639762 31304 raft_consensus.cc:482] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:53.639861 31304 raft_consensus.cc:3037] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:53.642233 31304 raft_consensus.cc:504] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.642938 31304 leader_election.cc:288] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [CANDIDATE]: Term 1 election: Requested vote from peers efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809), 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839)
I20260812 06:55:53.643517 30738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae7ff2e852e64157a53ff901d22378fb" candidate_uuid: "07a691a92dae43d180c025475bed6a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd7cbf9aeb0417eb8f3895be54521a6"
I20260812 06:55:53.643615 31167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae7ff2e852e64157a53ff901d22378fb" candidate_uuid: "07a691a92dae43d180c025475bed6a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348279277cca4a5ba93f85635ed27ff7"
I20260812 06:55:53.643760 30738 raft_consensus.cc:3037] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:53.643872 31167 raft_consensus.cc:3037] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:53.647321 30738 raft_consensus.cc:2454] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a691a92dae43d180c025475bed6a8b in term 1.
I20260812 06:55:53.647305 31167 raft_consensus.cc:2454] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a691a92dae43d180c025475bed6a8b in term 1.
I20260812 06:55:53.647809 30841 leader_election.cc:302] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a691a92dae43d180c025475bed6a8b, 348279277cca4a5ba93f85635ed27ff7; no voters:
I20260812 06:55:53.648204 31304 raft_consensus.cc:2781] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:53.648438 31304 raft_consensus.cc:686] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 1 LEADER]: Becoming Leader. State: Replica: 07a691a92dae43d180c025475bed6a8b, State: Running, Role: LEADER
I20260812 06:55:53.648901 31304 consensus_queue.cc:237] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:53.652999 30468 catalog_manager.cc:5495] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 07a691a92dae43d180c025475bed6a8b (127.26.226.2). New cstate: current_term: 1 leader_uuid: "07a691a92dae43d180c025475bed6a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:53.686103 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.674798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40988 (local address 127.26.226.2:42161)
0812 06:55:53.680487 (+ 5689us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.680492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.680509 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:53.680612 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.680616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.680695 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.680903 (+ 208us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:53.681313 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.682037 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.683523 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.684018 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.684235 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.684649 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.685323 (+ 674us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.685497 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.685609 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.685713 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5585,"thread_start_us":5486,"threads_started":1}
I20260812 06:55:53.702961 31285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.697424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:59603 (local address 127.26.226.3:42839)
0812 06:55:53.697626 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.697633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.697872 (+ 239us) server_negotiation.cc:407] Connection header received
0812 06:55:53.698167 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.698172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.698262 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.698360 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:53.698868 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.699615 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.701131 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.701575 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.701725 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.702193 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.702311 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.702416 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.702560 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.702642 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 06:55:53.703060 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.697039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:42839 (local address 127.26.226.2:59603)
0812 06:55:53.697758 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.697781 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.698009 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.698393 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.698397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.698410 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:53.698711 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.698720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.699768 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.699773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.700980 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.700993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.701780 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.701789 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.701929 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.701988 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.702419 (+ 431us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.702433 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.702534 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.702670 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":606,"threads_started":1}
I20260812 06:55:53.714737 31167 raft_consensus.cc:1261] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Refusing update from remote peer 07a691a92dae43d180c025475bed6a8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:53.715514 30738 raft_consensus.cc:1261] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Refusing update from remote peer 07a691a92dae43d180c025475bed6a8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:53.715862 31348 consensus_queue.cc:1035] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:53.716969 31348 consensus_queue.cc:1035] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:53.723892 31345 mvcc.cc:205] Tried to move back new op lower bound from 7317576719209603072 to 7317576718950211584. Current Snapshot: MvccSnapshot[applied={T|T < 7317576719209603072}]
I20260812 06:55:53.726058 31340 mvcc.cc:205] Tried to move back new op lower bound from 7317576719209603072 to 7317576718950211584. Current Snapshot: MvccSnapshot[applied={T|T < 7317576719209603072}]
I20260812 06:55:53.753348 31368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.747328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:42161 (local address 127.26.226.3:54979)
0812 06:55:53.747527 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.747549 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.747670 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.748139 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.748144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.748158 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:53.748350 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.748361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.749783 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.749788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.751171 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.751188 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.752128 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.752134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.752312 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.752358 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.752793 (+ 435us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.752805 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.752877 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.753042 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 06:55:53.754035 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.747682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:54979 (local address 127.26.226.2:42161)
0812 06:55:53.747840 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.747845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.747862 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:53.747922 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.747927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.748000 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.748111 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:53.748524 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.749600 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.751341 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.751885 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.751995 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.752531 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:53.752659 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:53.752739 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.753631 (+ 892us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.753723 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 06:55:53.755578 31345 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 06:55:55.384593 30468 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:55.384812 30468 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:55.389184 30468 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb: Sending DeleteTablet for 3 replicas of tablet ae7ff2e852e64157a53ff901d22378fb
I20260812 06:55:55.390105 30697 tablet_service.cc:1514] Processing DeleteTablet for tablet ae7ff2e852e64157a53ff901d22378fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:55 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:51016
I20260812 06:55:55.390599 30920 tablet_service.cc:1514] Processing DeleteTablet for tablet ae7ff2e852e64157a53ff901d22378fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:55 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:40982
I20260812 06:55:55.391062 31135 tablet_service.cc:1514] Processing DeleteTablet for tablet ae7ff2e852e64157a53ff901d22378fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:55 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:57750
I20260812 06:55:55.391744 31439 tablet_replica.cc:330] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6: stopping tablet replica
I20260812 06:55:55.392026 31439 raft_consensus.cc:2229] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:55.392364 31439 raft_consensus.cc:2258] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:55.393671 31439 ts_tablet_manager.cc:1905] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:55.400695 31442 tablet_replica.cc:330] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7: stopping tablet replica
I20260812 06:55:55.400967 31442 raft_consensus.cc:2229] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:55.401238 31442 raft_consensus.cc:2258] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:55.402160 31442 ts_tablet_manager.cc:1905] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:55.402643 31439 ts_tablet_manager.cc:1918] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:55.402897 31439 log.cc:1196] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/ae7ff2e852e64157a53ff901d22378fb
I20260812 06:55:55.403613 31439 ts_tablet_manager.cc:1939] T ae7ff2e852e64157a53ff901d22378fb P efd7cbf9aeb0417eb8f3895be54521a6: Deleting consensus metadata
I20260812 06:55:55.405206 30444 catalog_manager.cc:4841] TS efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809): tablet ae7ff2e852e64157a53ff901d22378fb (table test-table [id=db1141b60db7485299d531af22547698]) successfully deleted
I20260812 06:55:55.408578 31441 tablet_replica.cc:330] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b: stopping tablet replica
I20260812 06:55:55.408841 31442 ts_tablet_manager.cc:1918] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:55.408885 31441 raft_consensus.cc:2229] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:55.409042 31442 log.cc:1196] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/ae7ff2e852e64157a53ff901d22378fb
I20260812 06:55:55.409547 31442 ts_tablet_manager.cc:1939] T ae7ff2e852e64157a53ff901d22378fb P 348279277cca4a5ba93f85635ed27ff7: Deleting consensus metadata
I20260812 06:55:55.409564 31441 raft_consensus.cc:2258] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:55.410653 31441 ts_tablet_manager.cc:1905] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:55.411047 30443 catalog_manager.cc:4841] TS 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839): tablet ae7ff2e852e64157a53ff901d22378fb (table test-table [id=db1141b60db7485299d531af22547698]) successfully deleted
I20260812 06:55:55.418694 31441 ts_tablet_manager.cc:1918] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:55.418871 31441 log.cc:1196] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/ae7ff2e852e64157a53ff901d22378fb
I20260812 06:55:55.419371 31441 ts_tablet_manager.cc:1939] T ae7ff2e852e64157a53ff901d22378fb P 07a691a92dae43d180c025475bed6a8b: Deleting consensus metadata
I20260812 06:55:55.420634 30443 catalog_manager.cc:4841] TS 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161): tablet ae7ff2e852e64157a53ff901d22378fb (table test-table [id=db1141b60db7485299d531af22547698]) successfully deleted
Aug 12 06:55:55 dist-test-slave-t3q3 krb5kdc[30392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517755, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:55:55 dist-test-slave-t3q3 krb5kdc[30392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517755, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:55.460199 31457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:55.445200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.26.226.62:45859)
0812 06:55:55.445826 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:55.445834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:55.445870 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:55:55.446056 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:55.446060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:55.446124 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:55.446230 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:55.447706 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:55.448602 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:55.450099 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:55.450716 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:55.450822 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:55.454954 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:55.454959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:55.454962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:55.454997 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:55.457798 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:55.458413 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:55.458417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:55.458437 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:55.458587 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:55.458915 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:55.458918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:55.458920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:55.459230 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:55.459413 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:55.459655 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 06:55:55.459840 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":302,"threads_started":1}
I20260812 06:55:55.463614 30468 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:55.464324 30468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:55.471154 30697 tablet_service.cc:1467] Processing CreateTablet for tablet 052d540c1fd848f9bf027e29713d83b0 (DEFAULT_TABLE table=test-table [id=4b76171151c2458fba21fb3cb2b1ebb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:55.471199 30920 tablet_service.cc:1467] Processing CreateTablet for tablet 052d540c1fd848f9bf027e29713d83b0 (DEFAULT_TABLE table=test-table [id=4b76171151c2458fba21fb3cb2b1ebb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:55.471300 31135 tablet_service.cc:1467] Processing CreateTablet for tablet 052d540c1fd848f9bf027e29713d83b0 (DEFAULT_TABLE table=test-table [id=4b76171151c2458fba21fb3cb2b1ebb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:55.471648 30920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 052d540c1fd848f9bf027e29713d83b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:55.471653 30697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 052d540c1fd848f9bf027e29713d83b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:55.471707 31135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 052d540c1fd848f9bf027e29713d83b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:55.479616 31460 tablet_bootstrap.cc:492] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7: Bootstrap starting.
I20260812 06:55:55.480587 31458 tablet_bootstrap.cc:492] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b: Bootstrap starting.
I20260812 06:55:55.480871 31459 tablet_bootstrap.cc:492] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6: Bootstrap starting.
I20260812 06:55:55.481709 31460 tablet_bootstrap.cc:654] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:55.482160 31458 tablet_bootstrap.cc:654] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:55.483250 31459 tablet_bootstrap.cc:654] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:55.486001 31459 tablet_bootstrap.cc:492] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6: No bootstrap required, opened a new log
I20260812 06:55:55.486228 31459 ts_tablet_manager.cc:1397] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260812 06:55:55.486860 31459 raft_consensus.cc:348] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.487131 31459 raft_consensus.cc:374] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:55.487247 31459 raft_consensus.cc:729] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd7cbf9aeb0417eb8f3895be54521a6, State: Initialized, Role: FOLLOWER
I20260812 06:55:55.487504 31459 consensus_queue.cc:260] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.488446 31459 ts_tablet_manager.cc:1428] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 06:55:55.489315 31460 tablet_bootstrap.cc:492] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7: No bootstrap required, opened a new log
I20260812 06:55:55.489506 31460 ts_tablet_manager.cc:1397] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260812 06:55:55.490271 31460 raft_consensus.cc:348] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.490583 31460 raft_consensus.cc:374] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:55.490712 31458 tablet_bootstrap.cc:492] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b: No bootstrap required, opened a new log
I20260812 06:55:55.490725 31460 raft_consensus.cc:729] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348279277cca4a5ba93f85635ed27ff7, State: Initialized, Role: FOLLOWER
I20260812 06:55:55.490868 31458 ts_tablet_manager.cc:1397] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260812 06:55:55.490962 31460 consensus_queue.cc:260] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.491559 31460 ts_tablet_manager.cc:1428] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 06:55:55.491681 31458 raft_consensus.cc:348] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.492041 31458 raft_consensus.cc:374] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:55.493269 31458 raft_consensus.cc:729] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a691a92dae43d180c025475bed6a8b, State: Initialized, Role: FOLLOWER
I20260812 06:55:55.493561 31458 consensus_queue.cc:260] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.494124 31458 ts_tablet_manager.cc:1428] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 06:55:55.518033 31404 raft_consensus.cc:482] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:55.518292 31404 raft_consensus.cc:504] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.519019 31404 leader_election.cc:288] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809), 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839)
I20260812 06:55:55.519636 30738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052d540c1fd848f9bf027e29713d83b0" candidate_uuid: "07a691a92dae43d180c025475bed6a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" is_pre_election: true
I20260812 06:55:55.519666 31166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052d540c1fd848f9bf027e29713d83b0" candidate_uuid: "07a691a92dae43d180c025475bed6a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348279277cca4a5ba93f85635ed27ff7" is_pre_election: true
I20260812 06:55:55.519893 30738 raft_consensus.cc:2454] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a691a92dae43d180c025475bed6a8b in term 0.
I20260812 06:55:55.519928 31166 raft_consensus.cc:2454] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a691a92dae43d180c025475bed6a8b in term 0.
I20260812 06:55:55.520529 30844 leader_election.cc:302] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 07a691a92dae43d180c025475bed6a8b, 348279277cca4a5ba93f85635ed27ff7, efd7cbf9aeb0417eb8f3895be54521a6; no voters:
I20260812 06:55:55.520897 31404 raft_consensus.cc:2781] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:55.521035 31404 raft_consensus.cc:482] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:55.521154 31404 raft_consensus.cc:3037] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:55.523269 31404 raft_consensus.cc:504] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.523922 31404 leader_election.cc:288] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [CANDIDATE]: Term 1 election: Requested vote from peers efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809), 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839)
I20260812 06:55:55.524494 31166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052d540c1fd848f9bf027e29713d83b0" candidate_uuid: "07a691a92dae43d180c025475bed6a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348279277cca4a5ba93f85635ed27ff7"
I20260812 06:55:55.524526 30738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052d540c1fd848f9bf027e29713d83b0" candidate_uuid: "07a691a92dae43d180c025475bed6a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd7cbf9aeb0417eb8f3895be54521a6"
I20260812 06:55:55.524735 31166 raft_consensus.cc:3037] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:55.524770 30738 raft_consensus.cc:3037] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:55.527786 30738 raft_consensus.cc:2454] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a691a92dae43d180c025475bed6a8b in term 1.
I20260812 06:55:55.527786 31166 raft_consensus.cc:2454] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a691a92dae43d180c025475bed6a8b in term 1.
I20260812 06:55:55.528241 30844 leader_election.cc:302] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a691a92dae43d180c025475bed6a8b, efd7cbf9aeb0417eb8f3895be54521a6; no voters:
I20260812 06:55:55.528582 31404 raft_consensus.cc:2781] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:55.528771 31404 raft_consensus.cc:686] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 1 LEADER]: Becoming Leader. State: Replica: 07a691a92dae43d180c025475bed6a8b, State: Running, Role: LEADER
I20260812 06:55:55.529052 31404 consensus_queue.cc:237] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.531795 30468 catalog_manager.cc:5495] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 07a691a92dae43d180c025475bed6a8b (127.26.226.2). New cstate: current_term: 1 leader_uuid: "07a691a92dae43d180c025475bed6a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:55.556269 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:55.551238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41000 (local address 127.26.226.2:42161)
0812 06:55:55.551783 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:55.551787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:55.551807 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:55.551859 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:55.551862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:55.551920 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:55.552002 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:55.552418 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:55.553213 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:55.554497 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:55.554931 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:55.555026 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:55.555402 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:55.555578 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:55.555650 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:55.555794 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 06:55:55.555865 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":299,"threads_started":1}
I20260812 06:55:55.566004 30738 raft_consensus.cc:1261] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Refusing update from remote peer 07a691a92dae43d180c025475bed6a8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:55.566063 31166 raft_consensus.cc:1261] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Refusing update from remote peer 07a691a92dae43d180c025475bed6a8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:55.566639 31404 consensus_queue.cc:1035] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:55.566933 31429 consensus_queue.cc:1035] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:55.613945 30469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:55.614226 30469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:55.617112 30469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb: Sending DeleteTablet for 3 replicas of tablet 052d540c1fd848f9bf027e29713d83b0
I20260812 06:55:55.617928 30697 tablet_service.cc:1514] Processing DeleteTablet for tablet 052d540c1fd848f9bf027e29713d83b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:55 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:51016
I20260812 06:55:55.617904 30920 tablet_service.cc:1514] Processing DeleteTablet for tablet 052d540c1fd848f9bf027e29713d83b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:55 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:40982
I20260812 06:55:55.618093 31135 tablet_service.cc:1514] Processing DeleteTablet for tablet 052d540c1fd848f9bf027e29713d83b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:55 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:57750
I20260812 06:55:55.618242 31441 tablet_replica.cc:330] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b: stopping tablet replica
I20260812 06:55:55.618244 31439 tablet_replica.cc:330] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6: stopping tablet replica
I20260812 06:55:55.618430 31442 tablet_replica.cc:330] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7: stopping tablet replica
I20260812 06:55:55.618477 31441 raft_consensus.cc:2229] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:55.618484 31439 raft_consensus.cc:2229] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:55.618633 31442 raft_consensus.cc:2229] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:55.618698 31439 raft_consensus.cc:2258] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:55.618746 31441 raft_consensus.cc:2258] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:55.618826 31442 raft_consensus.cc:2258] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:55.619869 31442 ts_tablet_manager.cc:1905] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:55.620024 31439 ts_tablet_manager.cc:1905] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:55.620674 30469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 06:55:55.621063 31441 ts_tablet_manager.cc:1905] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 06:55:55.621439 30469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:55.626619 31439 ts_tablet_manager.cc:1918] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:55.626689 31442 ts_tablet_manager.cc:1918] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:55.626755 31439 log.cc:1196] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/052d540c1fd848f9bf027e29713d83b0
I20260812 06:55:55.626842 31442 log.cc:1196] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/052d540c1fd848f9bf027e29713d83b0
I20260812 06:55:55.627082 31439 ts_tablet_manager.cc:1939] T 052d540c1fd848f9bf027e29713d83b0 P efd7cbf9aeb0417eb8f3895be54521a6: Deleting consensus metadata
I20260812 06:55:55.627239 31442 ts_tablet_manager.cc:1939] T 052d540c1fd848f9bf027e29713d83b0 P 348279277cca4a5ba93f85635ed27ff7: Deleting consensus metadata
I20260812 06:55:55.627734 31441 ts_tablet_manager.cc:1918] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:55.627885 31441 log.cc:1196] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/052d540c1fd848f9bf027e29713d83b0
I20260812 06:55:55.628108 30444 catalog_manager.cc:4841] TS efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809): tablet 052d540c1fd848f9bf027e29713d83b0 (table test-table [id=4b76171151c2458fba21fb3cb2b1ebb8]) successfully deleted
I20260812 06:55:55.628319 31441 ts_tablet_manager.cc:1939] T 052d540c1fd848f9bf027e29713d83b0 P 07a691a92dae43d180c025475bed6a8b: Deleting consensus metadata
I20260812 06:55:55.628444 30443 catalog_manager.cc:4841] TS 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839): tablet 052d540c1fd848f9bf027e29713d83b0 (table test-table [id=4b76171151c2458fba21fb3cb2b1ebb8]) successfully deleted
I20260812 06:55:55.629381 30443 catalog_manager.cc:4841] TS 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161): tablet 052d540c1fd848f9bf027e29713d83b0 (table test-table [id=4b76171151c2458fba21fb3cb2b1ebb8]) successfully deleted
I20260812 06:55:55.632355 30697 tablet_service.cc:1467] Processing CreateTablet for tablet 14c4ffcef2d0492ab6846077f37e5a03 (DEFAULT_TABLE table=test-table [id=c33a861269734352937839bbd0a8d5c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:55.632617 30920 tablet_service.cc:1467] Processing CreateTablet for tablet 14c4ffcef2d0492ab6846077f37e5a03 (DEFAULT_TABLE table=test-table [id=c33a861269734352937839bbd0a8d5c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:55.632822 30697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14c4ffcef2d0492ab6846077f37e5a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:55.632925 31135 tablet_service.cc:1467] Processing CreateTablet for tablet 14c4ffcef2d0492ab6846077f37e5a03 (DEFAULT_TABLE table=test-table [id=c33a861269734352937839bbd0a8d5c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:55.633057 30920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14c4ffcef2d0492ab6846077f37e5a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:55.633354 31135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14c4ffcef2d0492ab6846077f37e5a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:55.638074 31460 tablet_bootstrap.cc:492] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7: Bootstrap starting.
I20260812 06:55:55.639422 31458 tablet_bootstrap.cc:492] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b: Bootstrap starting.
I20260812 06:55:55.639501 31460 tablet_bootstrap.cc:654] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:55.640316 31459 tablet_bootstrap.cc:492] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6: Bootstrap starting.
I20260812 06:55:55.641057 31458 tablet_bootstrap.cc:654] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:55.641825 31460 tablet_bootstrap.cc:492] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7: No bootstrap required, opened a new log
I20260812 06:55:55.641992 31459 tablet_bootstrap.cc:654] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:55.642323 31460 ts_tablet_manager.cc:1397] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 06:55:55.643513 31460 raft_consensus.cc:348] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.643867 31460 raft_consensus.cc:374] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:55.643975 31460 raft_consensus.cc:729] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348279277cca4a5ba93f85635ed27ff7, State: Initialized, Role: FOLLOWER
I20260812 06:55:55.644233 31459 tablet_bootstrap.cc:492] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6: No bootstrap required, opened a new log
I20260812 06:55:55.644415 31459 ts_tablet_manager.cc:1397] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 06:55:55.644367 31460 consensus_queue.cc:260] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.645097 31459 raft_consensus.cc:348] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.645422 31459 raft_consensus.cc:374] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:55.645594 31459 raft_consensus.cc:729] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd7cbf9aeb0417eb8f3895be54521a6, State: Initialized, Role: FOLLOWER
I20260812 06:55:55.645951 31459 consensus_queue.cc:260] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.646610 31459 ts_tablet_manager.cc:1428] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:55:55.646649 31460 ts_tablet_manager.cc:1428] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 06:55:55.648245 31458 tablet_bootstrap.cc:492] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b: No bootstrap required, opened a new log
I20260812 06:55:55.648444 31458 ts_tablet_manager.cc:1397] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.007s
I20260812 06:55:55.649278 31458 raft_consensus.cc:348] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.649585 31458 raft_consensus.cc:374] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:55.649722 31458 raft_consensus.cc:729] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a691a92dae43d180c025475bed6a8b, State: Initialized, Role: FOLLOWER
I20260812 06:55:55.650000 31458 consensus_queue.cc:260] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.650528 31458 ts_tablet_manager.cc:1428] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:55.659999 31464 raft_consensus.cc:482] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:55.660251 31464 raft_consensus.cc:504] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.660859 31464 leader_election.cc:288] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161), 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839)
I20260812 06:55:55.661370 30951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4ffcef2d0492ab6846077f37e5a03" candidate_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a691a92dae43d180c025475bed6a8b" is_pre_election: true
I20260812 06:55:55.661619 31166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4ffcef2d0492ab6846077f37e5a03" candidate_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348279277cca4a5ba93f85635ed27ff7" is_pre_election: true
I20260812 06:55:55.661677 30951 raft_consensus.cc:2454] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd7cbf9aeb0417eb8f3895be54521a6 in term 0.
I20260812 06:55:55.661913 31166 raft_consensus.cc:2454] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd7cbf9aeb0417eb8f3895be54521a6 in term 0.
I20260812 06:55:55.662228 30609 leader_election.cc:302] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a691a92dae43d180c025475bed6a8b, efd7cbf9aeb0417eb8f3895be54521a6; no voters:
I20260812 06:55:55.662556 31464 raft_consensus.cc:2781] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:55.662719 31464 raft_consensus.cc:482] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:55.662849 31464 raft_consensus.cc:3037] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:55.664955 31464 raft_consensus.cc:504] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.665499 31464 leader_election.cc:288] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [CANDIDATE]: Term 1 election: Requested vote from peers 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161), 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839)
I20260812 06:55:55.665994 30951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4ffcef2d0492ab6846077f37e5a03" candidate_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a691a92dae43d180c025475bed6a8b"
I20260812 06:55:55.666083 31166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4ffcef2d0492ab6846077f37e5a03" candidate_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348279277cca4a5ba93f85635ed27ff7"
I20260812 06:55:55.666281 31166 raft_consensus.cc:3037] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:55.666281 30951 raft_consensus.cc:3037] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:55.669330 31166 raft_consensus.cc:2454] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd7cbf9aeb0417eb8f3895be54521a6 in term 1.
I20260812 06:55:55.669325 30951 raft_consensus.cc:2454] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd7cbf9aeb0417eb8f3895be54521a6 in term 1.
I20260812 06:55:55.669770 30609 leader_election.cc:302] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a691a92dae43d180c025475bed6a8b, efd7cbf9aeb0417eb8f3895be54521a6; no voters:
I20260812 06:55:55.670085 31464 raft_consensus.cc:2781] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:55.670511 31464 raft_consensus.cc:686] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 LEADER]: Becoming Leader. State: Replica: efd7cbf9aeb0417eb8f3895be54521a6, State: Running, Role: LEADER
I20260812 06:55:55.670823 31464 consensus_queue.cc:237] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } }
I20260812 06:55:55.673468 30469 catalog_manager.cc:5495] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1). New cstate: current_term: 1 leader_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd7cbf9aeb0417eb8f3895be54521a6" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 37809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:55.701282 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:55.696037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.26.226.1:37809)
0812 06:55:55.696753 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:55.696757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:55.696777 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:55.696827 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:55.696830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:55.696891 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:55.696973 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:55.697357 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:55.698167 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:55.699539 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:55.699949 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:55.700043 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:55.700518 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:55.700620 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:55.700689 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:55.700842 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 06:55:55.700902 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":416,"threads_started":1}
I20260812 06:55:55.710211 31166 raft_consensus.cc:1261] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Refusing update from remote peer efd7cbf9aeb0417eb8f3895be54521a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:55.710220 30951 raft_consensus.cc:1261] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Refusing update from remote peer efd7cbf9aeb0417eb8f3895be54521a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:55.710790 31489 consensus_queue.cc:1035] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a691a92dae43d180c025475bed6a8b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:55.711211 31464 consensus_queue.cc:1035] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "348279277cca4a5ba93f85635ed27ff7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:55:55.736297 31411 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 06:55:57.327478 30467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:57.327696 30467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:57.331893 30467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bb8020277904ae7907b7bf4e6bb8abb: Sending DeleteTablet for 3 replicas of tablet 14c4ffcef2d0492ab6846077f37e5a03
I20260812 06:55:57.332731 30697 tablet_service.cc:1514] Processing DeleteTablet for tablet 14c4ffcef2d0492ab6846077f37e5a03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:57 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:51016
I20260812 06:55:57.333070 30920 tablet_service.cc:1514] Processing DeleteTablet for tablet 14c4ffcef2d0492ab6846077f37e5a03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:57 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:40982
I20260812 06:55:57.333066 31135 tablet_service.cc:1514] Processing DeleteTablet for tablet 14c4ffcef2d0492ab6846077f37e5a03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:57 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:57750
I20260812 06:55:57.333684 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30587
I20260812 06:55:57.346568 31538 tablet_replica.cc:330] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7: stopping tablet replica
I20260812 06:55:57.347121 31538 raft_consensus.cc:2229] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:57.347597 31538 raft_consensus.cc:2258] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:57.348304 31536 tablet_replica.cc:330] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b: stopping tablet replica
I20260812 06:55:57.348363 31534 tablet_replica.cc:330] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6: stopping tablet replica
I20260812 06:55:57.348694 31534 raft_consensus.cc:2229] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:57.349166 31534 raft_consensus.cc:2258] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:57.349175 31538 ts_tablet_manager.cc:1905] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:57.349629 31536 raft_consensus.cc:2229] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:57.350441 31534 ts_tablet_manager.cc:1905] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:57.350760 31536 raft_consensus.cc:2258] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:57.355245 31536 ts_tablet_manager.cc:1905] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:57.356294 31538 ts_tablet_manager.cc:1918] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:57.356491 31538 log.cc:1196] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/14c4ffcef2d0492ab6846077f37e5a03
I20260812 06:55:57.357105 31538 ts_tablet_manager.cc:1939] T 14c4ffcef2d0492ab6846077f37e5a03 P 348279277cca4a5ba93f85635ed27ff7: Deleting consensus metadata
I20260812 06:55:57.358513 30443 catalog_manager.cc:4841] TS 348279277cca4a5ba93f85635ed27ff7 (127.26.226.3:42839): tablet 14c4ffcef2d0492ab6846077f37e5a03 (table test-table [id=c33a861269734352937839bbd0a8d5c8]) successfully deleted
I20260812 06:55:57.361203 31536 ts_tablet_manager.cc:1918] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:57.361362 31536 log.cc:1196] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/14c4ffcef2d0492ab6846077f37e5a03
I20260812 06:55:57.361435 31534 ts_tablet_manager.cc:1918] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:57.361603 31534 log.cc:1196] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/14c4ffcef2d0492ab6846077f37e5a03
I20260812 06:55:57.361878 31536 ts_tablet_manager.cc:1939] T 14c4ffcef2d0492ab6846077f37e5a03 P 07a691a92dae43d180c025475bed6a8b: Deleting consensus metadata
I20260812 06:55:57.362164 31534 ts_tablet_manager.cc:1939] T 14c4ffcef2d0492ab6846077f37e5a03 P efd7cbf9aeb0417eb8f3895be54521a6: Deleting consensus metadata
I20260812 06:55:57.362999 30443 catalog_manager.cc:4841] TS 07a691a92dae43d180c025475bed6a8b (127.26.226.2:42161): tablet 14c4ffcef2d0492ab6846077f37e5a03 (table test-table [id=c33a861269734352937839bbd0a8d5c8]) successfully deleted
I20260812 06:55:57.363366 30444 catalog_manager.cc:4841] TS efd7cbf9aeb0417eb8f3895be54521a6 (127.26.226.1:37809): tablet 14c4ffcef2d0492ab6846077f37e5a03 (table test-table [id=c33a861269734352937839bbd0a8d5c8]) successfully deleted
Aug 12 06:55:57 dist-test-slave-t3q3 krb5kdc[30392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517755, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:55:57.374847 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:57.335067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51052 (local address 127.26.226.1:37809)
0812 06:55:57.355630 (+ 20563us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:57.355635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:57.355655 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:57.355712 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:57.355716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:57.355782 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:57.355924 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:57.357773 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.358936 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.360835 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.361507 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.361654 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:57.369115 (+ 7461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:57.369131 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:57.369148 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:57.369186 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:57.372265 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.372935 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.372943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.372945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.373101 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.373448 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.373451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.373453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.373863 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:57.374107 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:57.374379 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 06:55:57.374560 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20444,"thread_start_us":20327,"threads_started":1}
W20260812 06:55:57.375607 30797 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51052
W20260812 06:55:57.376327 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30587: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:55:57.376549 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 30587
I20260812 06:55:57.415294 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30812
Aug 12 06:55:57 dist-test-slave-t3q3 krb5kdc[30392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517755, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:55:57.430531 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:57.416132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.26.226.2:42161)
0812 06:55:57.416695 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:57.416700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:57.416717 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:57.416776 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:57.416780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:57.416840 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:57.416941 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:57.418287 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.419020 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.420304 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.420702 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.420806 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:57.425123 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:57.425154 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:57.425173 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:57.425214 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:57.427984 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.428682 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.428690 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.428692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.428834 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.429171 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.429174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.429176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.429539 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:57.429770 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:57.429975 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 06:55:57.430161 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":317,"threads_started":1}
W20260812 06:55:57.431257 31018 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41008
W20260812 06:55:57.431921 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30812: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:55:57.432281 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 30812
I20260812 06:55:57.465245 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31034
Aug 12 06:55:57 dist-test-slave-t3q3 krb5kdc[30392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517755, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:55:57.480978 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:57.465998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57756 (local address 127.26.226.3:42839)
0812 06:55:57.466578 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:57.466582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:57.466599 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:57.466654 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:57.466658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:57.466727 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:57.466820 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:57.468113 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.468932 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.470127 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.470527 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.470620 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:57.475552 (+ 4932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:57.475569 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:57.475586 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:57.475621 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:57.478472 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.479166 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.479172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.479174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.479302 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.479632 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.479636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.479638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.479979 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:57.480293 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:57.480512 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 06:55:57.480682 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":342,"threads_started":1}
W20260812 06:55:57.481670 31246 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57756
W20260812 06:55:57.482266 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31034: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:55:57.482432 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 31034
I20260812 06:55:57.518318 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30418
I20260812 06:55:57.519639 30533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:57.638226 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 30418
2026-08-12T06:55:57Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8915 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:57 dist-test-slave-t3q3 krb5kdc[31551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:57 dist-test-slave-t3q3 krb5kdc[31551](info): set up 2 sockets
Aug 12 06:55:57 dist-test-slave-t3q3 krb5kdc[31551](info): commencing operation
krb5kdc: starting...
W20260812 06:55:58.984439 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:55:59 dist-test-slave-t3q3 krb5kdc[31551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:55:59.219921 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:37463
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:37463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:59.570144 31570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:59.570433 31570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:59.570526 31570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:59.577790 31570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:59.577896 31570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:59.578017 31570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:59.578111 31570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:59.578193 31570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:59.587903 31570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:37463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:37463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:55:59.589319 31570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:59.590637 31570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:59.602814 31578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:59.602885 31579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:59.602901 31582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:59.603711 31570 server_base.cc:1034] running on GCE node
I20260812 06:55:59.604662 31570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:59.607163 31570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:59.608505 31570 hybrid_clock.cc:648] HybridClock initialized: now 1786517759608402 us; error 88 us; skew 500 ppm
I20260812 06:55:59.608848 31570 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:59 dist-test-slave-t3q3 krb5kdc[31551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517759, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:59.611933 31570 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:55:59.613469 31570 webserver.cc:460] Webserver started at http://127.26.226.62:33809/ using document root <none> and password file <none>
I20260812 06:55:59.614120 31570 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:59.614221 31570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:59.614472 31570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:59.617471 31570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "a9ec912afb5a4067a182b812835420a0"
format_stamp: "Formatted at 2026-08-12 06:55:59 on dist-test-slave-t3q3"
server_key: "fad32943738b67156bda54db4f805f76"
server_key_iv: "8bb8b4672b53e27321267f5593fc8a2c"
server_key_version: "encryptionkey@0"
I20260812 06:55:59.618188 31570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "a9ec912afb5a4067a182b812835420a0"
format_stamp: "Formatted at 2026-08-12 06:55:59 on dist-test-slave-t3q3"
server_key: "fad32943738b67156bda54db4f805f76"
server_key_iv: "8bb8b4672b53e27321267f5593fc8a2c"
server_key_version: "encryptionkey@0"
I20260812 06:55:59.623417 31570 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 06:55:59.626801 31592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:59.628153 31570 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:59.628343 31570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "a9ec912afb5a4067a182b812835420a0"
format_stamp: "Formatted at 2026-08-12 06:55:59 on dist-test-slave-t3q3"
server_key: "fad32943738b67156bda54db4f805f76"
server_key_iv: "8bb8b4672b53e27321267f5593fc8a2c"
server_key_version: "encryptionkey@0"
I20260812 06:55:59.628547 31570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:59.646217 31570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:59.650404 31570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:59.650738 31570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:59.670253 31570 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:37463
I20260812 06:55:59.670277 31672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:37463 every 8 connection(s)
I20260812 06:55:59.671782 31570 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:59.673689 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 31570
I20260812 06:55:59.673954 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:55:59.674844 27528 external_mini_cluster.cc:1428] Setting key d0f9036959a14d3f41f07ef165aa755c
I20260812 06:55:59.676862 31674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 06:55:59 dist-test-slave-t3q3 krb5kdc[31551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:55:59.691254 31674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0: Bootstrap starting.
I20260812 06:55:59.694880 31674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:59.695988 31674 log.cc:824] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:59.698709 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:59.677224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.26.226.62:37463)
0812 06:55:59.678047 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:59.678059 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:59.678102 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:55:59.679751 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:59.679773 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:59.680238 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:59.680593 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:59.682436 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:59.683788 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:59.685587 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:59.686245 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:59.686381 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:59.691121 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:59.691130 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:59.691145 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:59.691183 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:59.694433 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:59.695231 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:59.695239 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:59.695249 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:59.695430 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:59.695865 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:59.695869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:59.695871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:59.696505 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:59.696885 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:59.697293 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 06:55:59.697743 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":394,"threads_started":1}
I20260812 06:55:59.699182 31674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0: No bootstrap required, opened a new log
I20260812 06:55:59.707788 31674 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec912afb5a4067a182b812835420a0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 37463 } }
I20260812 06:55:59.708249 31674 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:59.708381 31674 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ec912afb5a4067a182b812835420a0, State: Initialized, Role: FOLLOWER
I20260812 06:55:59.708957 31674 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec912afb5a4067a182b812835420a0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 37463 } }
I20260812 06:55:59.709203 31674 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:59.709350 31674 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:59.709549 31674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:59.712210 31674 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec912afb5a4067a182b812835420a0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 37463 } }
I20260812 06:55:59.712725 31674 leader_election.cc:302] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9ec912afb5a4067a182b812835420a0; no voters:
I20260812 06:55:59.713375 31674 leader_election.cc:288] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:59.713521 31682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:59.714226 31682 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [term 1 LEADER]: Becoming Leader. State: Replica: a9ec912afb5a4067a182b812835420a0, State: Running, Role: LEADER
I20260812 06:55:59.714774 31682 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec912afb5a4067a182b812835420a0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 37463 } }
I20260812 06:55:59.715533 31674 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:59.717751 31684 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9ec912afb5a4067a182b812835420a0. Latest consensus state: current_term: 1 leader_uuid: "a9ec912afb5a4067a182b812835420a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec912afb5a4067a182b812835420a0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 37463 } } }
I20260812 06:55:59.717964 31683 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9ec912afb5a4067a182b812835420a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec912afb5a4067a182b812835420a0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 37463 } } }
I20260812 06:55:59.718255 31684 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:59.718279 31683 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:59.720528 31695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:59.725713 31695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:59.735267 31695 catalog_manager.cc:1347] Generated new cluster ID: 656b41e2f77a4b2bb5a9bdb54b88ed85
I20260812 06:55:59.735419 31695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:59.764797 31695 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:59.766191 31695 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:59.779513 31695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9ec912afb5a4067a182b812835420a0: Generated new TSK 0
I20260812 06:55:59.780452 31695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:55:59.859208 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:37463
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:00.208603 31715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:00.208905 31715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:00.209102 31715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:00.216734 31715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:00.216885 31715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:00.217114 31715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:00.227916 31715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:37463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:00.229318 31715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:00.230654 31715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:00.244182 31721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:00.244342 31726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:00.244360 31723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:00.245318 31715 server_base.cc:1034] running on GCE node
I20260812 06:56:00.246032 31715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:00.247002 31715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:00.248222 31715 hybrid_clock.cc:648] HybridClock initialized: now 1786517760248180 us; error 56 us; skew 500 ppm
I20260812 06:56:00.248558 31715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:00 dist-test-slave-t3q3 krb5kdc[31551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517760, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:00.251909 31715 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:56:00.253459 31715 webserver.cc:460] Webserver started at http://127.26.226.1:34651/ using document root <none> and password file <none>
I20260812 06:56:00.254158 31715 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:00.254264 31715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:00.254532 31715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:00.257563 31715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "7ffd4b714627468b8e00af30ef60c4b2"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-t3q3"
server_key: "56341cb8c099f8c77b3e9ac99bbb8669"
server_key_iv: "6052210babb1b248c954d6ea87f45d28"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.258340 31715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "7ffd4b714627468b8e00af30ef60c4b2"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-t3q3"
server_key: "56341cb8c099f8c77b3e9ac99bbb8669"
server_key_iv: "6052210babb1b248c954d6ea87f45d28"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.263567 31715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 06:56:00.267306 31735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:00.268811 31715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:00.268960 31715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "7ffd4b714627468b8e00af30ef60c4b2"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-t3q3"
server_key: "56341cb8c099f8c77b3e9ac99bbb8669"
server_key_iv: "6052210babb1b248c954d6ea87f45d28"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.269140 31715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:00.300382 31715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:00.305315 31715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:00.305573 31715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:00.306558 31715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:00.308007 31715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:00.308133 31715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260812 06:56:00.308238 31715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:00.308293 31715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:00.343521 31715 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:38865
I20260812 06:56:00.343575 31927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:38865 every 8 connection(s)
I20260812 06:56:00.344877 31715 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:00.353472 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 31715
I20260812 06:56:00.353729 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:00.354363 27528 external_mini_cluster.cc:1428] Setting key 7c1e3692eab3d2ed5114b0e3b191ac43
Aug 12 06:56:00 dist-test-slave-t3q3 krb5kdc[31551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517760, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:00.366567 31930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:00.347461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:53651 (local address 127.26.226.62:37463)
0812 06:56:00.347949 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:00.347954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:00.348766 (+ 812us) server_negotiation.cc:407] Connection header received
0812 06:56:00.350629 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:00.350634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:00.350732 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:00.350852 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:00.352439 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.353338 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.355162 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.355662 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.355772 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:00.359421 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:00.359430 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:00.359438 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:00.359486 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:00.363174 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:00.363923 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:00.363931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:00.363937 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:00.364119 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:00.364559 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:00.364566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:00.364571 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:00.364970 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:00.365113 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:00.366132 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0812 06:56:00.366312 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":271,"threads_started":1}
I20260812 06:56:00.368222 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:00.347846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:37463 (local address 127.26.226.1:53651)
0812 06:56:00.348598 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:00.348642 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:00.350352 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:00.351096 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:00.351103 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:00.351625 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:00.352266 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:00.352286 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.353510 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.353514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:00.354962 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:00.354980 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.355914 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.355922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:00.356264 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:00.357022 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:00.357044 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:00.359188 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:00.363329 (+ 4141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:00.363335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:00.363347 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:00.363785 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:00.364236 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:00.364240 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:00.364242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:00.364435 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:00.365241 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:00.365250 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:00.365708 (+ 458us) client_negotiation.cc:769] Sending connection context
0812 06:56:00.366036 (+ 328us) client_negotiation.cc:240] Negotiation successful
0812 06:56:00.366663 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":351,"threads_started":1}
I20260812 06:56:00.369910 31928 heartbeater.cc:347] Connected to a master server at 127.26.226.62:37463
I20260812 06:56:00.370224 31928 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:00.370944 31928 heartbeater.cc:511] Master 127.26.226.62:37463 requested a full tablet report, sending...
I20260812 06:56:00.373119 31619 ts_manager.cc:194] Registered new tserver with Master: 7ffd4b714627468b8e00af30ef60c4b2 (127.26.226.1:38865)
I20260812 06:56:00.374632 31619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:53651
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:56:00.405324 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:37463
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:00.752552 31936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:00.752902 31936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:00.753084 31936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:00.760489 31936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:00.760637 31936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:00.760831 31936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:00.770712 31936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:37463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:00.772167 31936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:00.773722 31936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:00.787436 31943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:00.787544 31942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:00.787570 31945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:00.788450 31936 server_base.cc:1034] running on GCE node
I20260812 06:56:00.789011 31936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:00.789986 31936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:00.791234 31936 hybrid_clock.cc:648] HybridClock initialized: now 1786517760791185 us; error 57 us; skew 500 ppm
I20260812 06:56:00.791566 31936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:00 dist-test-slave-t3q3 krb5kdc[31551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517760, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:00.794989 31936 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:56:00.796725 31936 webserver.cc:460] Webserver started at http://127.26.226.2:33249/ using document root <none> and password file <none>
I20260812 06:56:00.797470 31936 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:00.797631 31936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:00.797978 31936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:00.801258 31936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "db35bd60847e48a98206b0203c23ccad"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-t3q3"
server_key: "8c00237a9a9248eaae5dfcbc0346531e"
server_key_iv: "f3b5a71777326009bf175b6a0206b56e"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.802129 31936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "db35bd60847e48a98206b0203c23ccad"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-t3q3"
server_key: "8c00237a9a9248eaae5dfcbc0346531e"
server_key_iv: "f3b5a71777326009bf175b6a0206b56e"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.808171 31936 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 06:56:00.811872 31954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:00.813308 31936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:00.813505 31936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "db35bd60847e48a98206b0203c23ccad"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-t3q3"
server_key: "8c00237a9a9248eaae5dfcbc0346531e"
server_key_iv: "f3b5a71777326009bf175b6a0206b56e"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.813743 31936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:00.839262 31936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:00.843940 31936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:00.844291 31936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:00.845335 31936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:00.846760 31936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:00.846871 31936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:00.847026 31936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:00.847103 31936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:00.882292 31936 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:36559
I20260812 06:56:00.882344 32133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:36559 every 8 connection(s)
I20260812 06:56:00.883642 31936 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:00.888587 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 31936
I20260812 06:56:00.888787 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:00.889251 27528 external_mini_cluster.cc:1428] Setting key a62a0950b0b862c08477d696296c7934
Aug 12 06:56:00 dist-test-slave-t3q3 krb5kdc[31551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517760, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:00.907513 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:00.886449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:55829 (local address 127.26.226.62:37463)
0812 06:56:00.886914 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:00.886918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:00.887900 (+ 982us) server_negotiation.cc:407] Connection header received
0812 06:56:00.890080 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:00.890086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:00.890171 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:00.890301 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:00.892099 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.893435 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.895013 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.895777 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.895928 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:00.900222 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:00.900232 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:00.900240 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:00.900292 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:00.904128 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:00.904928 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:00.904937 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:00.904943 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:00.905123 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:00.905585 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:00.905591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:00.905601 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:00.905923 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:00.906103 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:00.906848 (+ 745us) server_negotiation.cc:299] Negotiation successful
0812 06:56:00.907092 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":243,"threads_started":1}
I20260812 06:56:00.908638 32139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:00.886993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:37463 (local address 127.26.226.2:55829)
0812 06:56:00.887712 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:00.887757 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:00.889693 (+ 1936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:00.890612 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:00.890624 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:00.891328 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:00.891906 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:00.891924 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.893602 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.893607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:00.894855 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:00.894867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.895992 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.895997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:00.896297 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:00.897033 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:00.897056 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:00.899890 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:00.904287 (+ 4397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:00.904294 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:00.904309 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:00.904786 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:00.905243 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:00.905246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:00.905248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:00.905447 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:00.906082 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:00.906091 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:00.906460 (+ 369us) client_negotiation.cc:769] Sending connection context
0812 06:56:00.906761 (+ 301us) client_negotiation.cc:240] Negotiation successful
0812 06:56:00.907353 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":276,"threads_started":1}
I20260812 06:56:00.910586 32134 heartbeater.cc:347] Connected to a master server at 127.26.226.62:37463
I20260812 06:56:00.910938 32134 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:00.911737 32134 heartbeater.cc:511] Master 127.26.226.62:37463 requested a full tablet report, sending...
I20260812 06:56:00.913532 31619 ts_manager.cc:194] Registered new tserver with Master: db35bd60847e48a98206b0203c23ccad (127.26.226.2:36559)
I20260812 06:56:00.914835 31619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:55829
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:56:00.946966 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:37463
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:01.298727 32143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:01.299050 32143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:01.299238 32143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:01.309122 32143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:01.309260 32143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:01.309474 32143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:01.320288 32143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:37463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:01.321692 32143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:01.323158 32143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:01.340853 32150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:01.340893 32151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:01.340989 32153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:01.342020 32143 server_base.cc:1034] running on GCE node
I20260812 06:56:01.342769 32143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:01.343977 32143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:01.345244 32143 hybrid_clock.cc:648] HybridClock initialized: now 1786517761345173 us; error 59 us; skew 500 ppm
I20260812 06:56:01.345590 32143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:01 dist-test-slave-t3q3 krb5kdc[31551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517761, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:01.349041 32143 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:56:01.350657 32143 webserver.cc:460] Webserver started at http://127.26.226.3:38437/ using document root <none> and password file <none>
I20260812 06:56:01.351804 32143 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:01.351917 32143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:01.352300 32143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:01.355455 32143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "ebbf1651ec7340a78f360c4dc0e6f645"
format_stamp: "Formatted at 2026-08-12 06:56:01 on dist-test-slave-t3q3"
server_key: "364f1a5e09d39577592a539ccad0cae8"
server_key_iv: "210dcdef7ccb79cd04178c48c8e4b422"
server_key_version: "encryptionkey@0"
I20260812 06:56:01.356362 32143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "ebbf1651ec7340a78f360c4dc0e6f645"
format_stamp: "Formatted at 2026-08-12 06:56:01 on dist-test-slave-t3q3"
server_key: "364f1a5e09d39577592a539ccad0cae8"
server_key_iv: "210dcdef7ccb79cd04178c48c8e4b422"
server_key_version: "encryptionkey@0"
I20260812 06:56:01.362386 32143 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 06:56:01.366245 32169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:01.367659 32143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:56:01.367848 32143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "ebbf1651ec7340a78f360c4dc0e6f645"
format_stamp: "Formatted at 2026-08-12 06:56:01 on dist-test-slave-t3q3"
server_key: "364f1a5e09d39577592a539ccad0cae8"
server_key_iv: "210dcdef7ccb79cd04178c48c8e4b422"
server_key_version: "encryptionkey@0"
I20260812 06:56:01.368321 32143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:01.377595 31928 heartbeater.cc:503] Master 127.26.226.62:37463 was elected leader, sending a full tablet report...
I20260812 06:56:01.390820 32143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:01.395507 32143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:01.395812 32143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:01.396908 32143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:01.398316 32143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:01.398406 32143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:01.398492 32143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:01.398545 32143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:01.437933 32143 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:39253
I20260812 06:56:01.438027 32362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:39253 every 8 connection(s)
I20260812 06:56:01.439296 32143 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:01.441260 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 32143
I20260812 06:56:01.441488 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:01.442014 27528 external_mini_cluster.cc:1428] Setting key 1c65307423f9bf5d730079b6e0fae0c2
Aug 12 06:56:01 dist-test-slave-t3q3 krb5kdc[31551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517761, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:01.461439 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.442042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:46077 (local address 127.26.226.62:37463)
0812 06:56:01.442573 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.442580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.443784 (+ 1204us) server_negotiation.cc:407] Connection header received
0812 06:56:01.446523 (+ 2739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.446528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.446608 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.446698 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:01.448285 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.449023 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.450531 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.450924 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.451028 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.454625 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:01.454630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:01.454633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:01.454669 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:01.458121 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.459038 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.459043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.459045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.459186 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.459696 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.459700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.459702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.459991 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:01.460227 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.460977 (+ 750us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.461149 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":247,"threads_started":1}
I20260812 06:56:01.462466 32367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.442654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:37463 (local address 127.26.226.3:46077)
0812 06:56:01.443579 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:01.443653 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:01.446269 (+ 2616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:01.446912 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:01.446920 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:01.447474 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:01.448050 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.448121 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.449158 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.449162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.450399 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.450411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.451162 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.451167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.451362 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.452011 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:01.452041 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:01.454411 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:01.458311 (+ 3900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:01.458323 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:01.458349 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:01.458842 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:01.459385 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:01.459388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:01.459390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:01.459591 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:01.460232 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:01.460239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:01.460585 (+ 346us) client_negotiation.cc:769] Sending connection context
0812 06:56:01.460827 (+ 242us) client_negotiation.cc:240] Negotiation successful
0812 06:56:01.461243 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":368,"threads_started":1}
I20260812 06:56:01.463922 32364 heartbeater.cc:347] Connected to a master server at 127.26.226.62:37463
I20260812 06:56:01.464290 32364 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:01.465054 32364 heartbeater.cc:511] Master 127.26.226.62:37463 requested a full tablet report, sending...
I20260812 06:56:01.466482 31619 ts_manager.cc:194] Registered new tserver with Master: ebbf1651ec7340a78f360c4dc0e6f645 (127.26.226.3:39253)
I20260812 06:56:01.467267 31619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:46077
I20260812 06:56:01.470742 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 06:56:01.515655 32366 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:01.512984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.26.226.62:37463)
0812 06:56:01.513245 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.513253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.513688 (+ 435us) server_negotiation.cc:407] Connection header received
0812 06:56:01.513838 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.513842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.513925 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.514017 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:01.515364 (+ 1347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37636: BlockingRecv error: recv got EOF from 127.0.0.1:37636 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:56:01.519505 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31715
W20260812 06:56:01.522481 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31715: Not authorized: Client connection negotiation failed: client connection to 127.26.226.1:38865: server requires authentication, but client does not have Kerberos credentials available
I20260812 06:56:01.522702 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 31715
I20260812 06:56:01.545917 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31936
W20260812 06:56:01.548863 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31936: Not authorized: Client connection negotiation failed: client connection to 127.26.226.2:36559: server requires authentication, but client does not have Kerberos credentials available
I20260812 06:56:01.549044 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 31936
I20260812 06:56:01.576673 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32143
W20260812 06:56:01.580114 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32143: Not authorized: Client connection negotiation failed: client connection to 127.26.226.3:39253: server requires authentication, but client does not have Kerberos credentials available
I20260812 06:56:01.580303 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 32143
I20260812 06:56:01.605154 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31570
I20260812 06:56:01.606463 31665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:01.720480 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 31570
2026-08-12T06:56:01Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4078 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:03 dist-test-slave-t3q3 krb5kdc[32387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:03 dist-test-slave-t3q3 krb5kdc[32387](info): set up 2 sockets
Aug 12 06:56:03 dist-test-slave-t3q3 krb5kdc[32387](info): commencing operation
krb5kdc: starting...
W20260812 06:56:04.409965 27528 mini_kdc.cc:121] Time spent starting KDC: real 2.638s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:56:04 dist-test-slave-t3q3 krb5kdc[32387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:56:04.655821 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33251
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:36475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:33251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:05.009835 32405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:05.010159 32405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:05.010264 32405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:05.017704 32405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:05.017833 32405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:05.017882 32405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:05.017939 32405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:05.017990 32405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:05.028497 32405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:36475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:33251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:05.029932 32405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:05.031267 32405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:05.043787 32416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:05.043828 32418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:05.043745 32422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:05.045079 32405 server_base.cc:1034] running on GCE node
I20260812 06:56:05.045956 32405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:05.047410 32405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:05.048645 32405 hybrid_clock.cc:648] HybridClock initialized: now 1786517765048602 us; error 41 us; skew 500 ppm
I20260812 06:56:05.049023 32405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:05 dist-test-slave-t3q3 krb5kdc[32387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517765, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:05.056350 32405 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:56:05.058000 32405 webserver.cc:460] Webserver started at http://127.26.226.62:38389/ using document root <none> and password file <none>
I20260812 06:56:05.058729 32405 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:05.058885 32405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:05.059226 32405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:05.062417 32405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "d7b527aa61d643598e4806cca7bd5e1d"
format_stamp: "Formatted at 2026-08-12 06:56:05 on dist-test-slave-t3q3"
server_key: "5a024350f8baeee1237d6ad5e4b7b726"
server_key_iv: "7a11a42d9304cdadc3620cf53f3cbd8c"
server_key_version: "encryptionkey@0"
I20260812 06:56:05.063288 32405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "d7b527aa61d643598e4806cca7bd5e1d"
format_stamp: "Formatted at 2026-08-12 06:56:05 on dist-test-slave-t3q3"
server_key: "5a024350f8baeee1237d6ad5e4b7b726"
server_key_iv: "7a11a42d9304cdadc3620cf53f3cbd8c"
server_key_version: "encryptionkey@0"
I20260812 06:56:05.068699 32405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 06:56:05.072182 32440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:05.073519 32405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:05.073712 32405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "d7b527aa61d643598e4806cca7bd5e1d"
format_stamp: "Formatted at 2026-08-12 06:56:05 on dist-test-slave-t3q3"
server_key: "5a024350f8baeee1237d6ad5e4b7b726"
server_key_iv: "7a11a42d9304cdadc3620cf53f3cbd8c"
server_key_version: "encryptionkey@0"
I20260812 06:56:05.073915 32405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:05.085990 32405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:05.090119 32405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:05.090433 32405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:05.109779 32405 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:33251
I20260812 06:56:05.109815 32517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:33251 every 8 connection(s)
I20260812 06:56:05.111231 32405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:05.114957 32518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:05.120759 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 32405
I20260812 06:56:05.120998 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:05.121628 27528 external_mini_cluster.cc:1428] Setting key 7028697ad290c4cb095740ffce9d9d0c
I20260812 06:56:05.126053 32518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d: Bootstrap starting.
I20260812 06:56:05.129850 32518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:05.131091 32518 log.cc:824] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:05.134294 32518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d: No bootstrap required, opened a new log
Aug 12 06:56:05 dist-test-slave-t3q3 krb5kdc[32387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:05.145591 32518 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b527aa61d643598e4806cca7bd5e1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33251 } }
I20260812 06:56:05.146184 32518 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:05.146306 32518 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b527aa61d643598e4806cca7bd5e1d, State: Initialized, Role: FOLLOWER
I20260812 06:56:05.147014 32518 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b527aa61d643598e4806cca7bd5e1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33251 } }
I20260812 06:56:05.147301 32518 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:05.147454 32518 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:05.147578 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:05.123533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40630 (local address 127.26.226.62:33251)
0812 06:56:05.124420 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:05.124432 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:05.124473 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:56:05.126157 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:05.126179 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:05.126614 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:05.126933 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:05.128909 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:05.130260 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:05.132082 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:05.132851 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:05.133008 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:05.137722 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:05.137734 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:05.137759 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:05.137810 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:05.141939 (+ 4129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:05.142669 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:05.142683 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:05.142691 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:05.142904 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:05.143354 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:05.143362 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:05.143368 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:05.144392 (+ 1024us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:05.144740 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:05.145423 (+ 683us) server_negotiation.cc:299] Negotiation successful
0812 06:56:05.146134 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":433,"threads_started":1}
I20260812 06:56:05.147687 32518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:05.150527 32518 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b527aa61d643598e4806cca7bd5e1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33251 } }
I20260812 06:56:05.151185 32518 leader_election.cc:302] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7b527aa61d643598e4806cca7bd5e1d; no voters:
I20260812 06:56:05.151914 32518 leader_election.cc:288] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:05.153108 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:05.152228 32526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:05.155972 32518 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:05.159718 32526 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [term 1 LEADER]: Becoming Leader. State: Replica: d7b527aa61d643598e4806cca7bd5e1d, State: Running, Role: LEADER
I20260812 06:56:05.160409 32526 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b527aa61d643598e4806cca7bd5e1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33251 } }
I20260812 06:56:05.163926 32527 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7b527aa61d643598e4806cca7bd5e1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b527aa61d643598e4806cca7bd5e1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33251 } } }
I20260812 06:56:05.164371 32527 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:05.169247 32528 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7b527aa61d643598e4806cca7bd5e1d. Latest consensus state: current_term: 1 leader_uuid: "d7b527aa61d643598e4806cca7bd5e1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b527aa61d643598e4806cca7bd5e1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33251 } } }
I20260812 06:56:05.170131 32528 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:05.172618 32536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:05.179840 32536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:05.195568 32536 catalog_manager.cc:1347] Generated new cluster ID: bb11e420c0b945d6aeb443f70996f8a6
I20260812 06:56:05.195742 32536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:05.226001 32536 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:05.228577 32536 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:05.241356 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:05.230746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.26.226.62:33251)
0812 06:56:05.230934 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:05.230942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:05.231602 (+ 660us) server_negotiation.cc:407] Connection header received
0812 06:56:05.231861 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:05.231867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:05.231946 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:05.232022 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:05.233478 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:05.234568 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:05.237176 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:05.237907 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:05.238011 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:05.239603 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:05.239610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:05.239615 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:05.240449 (+ 834us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:05.240533 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:05.240566 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:05.240651 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:05.240886 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 06:56:05.240988 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:56:05.249925 32536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7b527aa61d643598e4806cca7bd5e1d: Generated new TSK 0
I20260812 06:56:05.251142 32536 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:05.261982 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32405
I20260812 06:56:05.264048 32511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:05.357267 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 32405
2026-08-12T06:56:05Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (3634 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:05 dist-test-slave-t3q3 krb5kdc[32570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:05 dist-test-slave-t3q3 krb5kdc[32570](info): set up 2 sockets
Aug 12 06:56:05 dist-test-slave-t3q3 krb5kdc[32570](info): commencing operation
krb5kdc: starting...
W20260812 06:56:06.695667 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:56:06 dist-test-slave-t3q3 krb5kdc[32570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:56:06.940328 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:43301
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:43301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:07.283656 32591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:07.283964 32591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:07.284211 32591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:07.291615 32591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:07.291736 32591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:07.291821 32591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:07.291909 32591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:07.292003 32591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:07.302070 32591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:43301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:43301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:07.303481 32591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:07.304888 32591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:07.316968 32599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:07.317034 32603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:07.317013 32600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:07.317742 32591 server_base.cc:1034] running on GCE node
I20260812 06:56:07.318578 32591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:07.320053 32591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:07.321380 32591 hybrid_clock.cc:648] HybridClock initialized: now 1786517767321314 us; error 52 us; skew 500 ppm
I20260812 06:56:07.321789 32591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:07 dist-test-slave-t3q3 krb5kdc[32570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517767, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:07.325304 32591 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:56:07.326944 32591 webserver.cc:460] Webserver started at http://127.26.226.62:36525/ using document root <none> and password file <none>
I20260812 06:56:07.327715 32591 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:07.327914 32591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:07.328296 32591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:07.331404 32591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "9d407868d0304962868fbdd80da6a323"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-t3q3"
server_key: "86058607e970d65571398ef62ebd2ceb"
server_key_iv: "2d89969d89031effbad1a54b2f66c9f1"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.332269 32591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "9d407868d0304962868fbdd80da6a323"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-t3q3"
server_key: "86058607e970d65571398ef62ebd2ceb"
server_key_iv: "2d89969d89031effbad1a54b2f66c9f1"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.337574 32591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 06:56:07.341156 32614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:07.342523 32591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:56:07.342726 32591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "9d407868d0304962868fbdd80da6a323"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-t3q3"
server_key: "86058607e970d65571398ef62ebd2ceb"
server_key_iv: "2d89969d89031effbad1a54b2f66c9f1"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.342936 32591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:07.363307 32591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:07.367609 32591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:07.367946 32591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:07.387341 32591 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:43301
I20260812 06:56:07.387383 32694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:43301 every 8 connection(s)
I20260812 06:56:07.388833 32591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:07.392732 32695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:07.394970 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 32591
I20260812 06:56:07.395197 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:07.395861 27528 external_mini_cluster.cc:1428] Setting key ac2fac2dc35afc7f5b13a4dc049706c1
I20260812 06:56:07.405694 32695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323: Bootstrap starting.
I20260812 06:56:07.409332 32695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:07.410375 32695 log.cc:824] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:56:07 dist-test-slave-t3q3 krb5kdc[32570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:07.413429 32695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323: No bootstrap required, opened a new log
I20260812 06:56:07.420029 32698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:07.398077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.26.226.62:43301)
0812 06:56:07.398945 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:07.398960 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:07.399006 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:56:07.401363 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:07.401394 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:07.401898 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:07.402361 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:07.403672 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:07.405644 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:07.406965 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:07.408084 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:07.408289 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:07.412512 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:07.412520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:07.412534 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:07.412574 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:07.415743 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:07.416556 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:07.416563 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:07.416569 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:07.416748 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:07.417166 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:07.417173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:07.417176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:07.417747 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:07.418108 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:07.418603 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 06:56:07.419007 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":363,"threads_started":1}
I20260812 06:56:07.421713 32695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d407868d0304962868fbdd80da6a323" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43301 } }
I20260812 06:56:07.422076 32695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:07.422211 32695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d407868d0304962868fbdd80da6a323, State: Initialized, Role: FOLLOWER
I20260812 06:56:07.422875 32695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d407868d0304962868fbdd80da6a323" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43301 } }
I20260812 06:56:07.423157 32695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:07.423292 32695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:07.423411 32695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:07.425915 32695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d407868d0304962868fbdd80da6a323" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43301 } }
I20260812 06:56:07.426481 32695 leader_election.cc:302] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d407868d0304962868fbdd80da6a323; no voters:
I20260812 06:56:07.427085 32695 leader_election.cc:288] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:07.427235 32701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:07.427992 32701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [term 1 LEADER]: Becoming Leader. State: Replica: 9d407868d0304962868fbdd80da6a323, State: Running, Role: LEADER
I20260812 06:56:07.428546 32701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d407868d0304962868fbdd80da6a323" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43301 } }
I20260812 06:56:07.429184 32695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:07.431695 32702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d407868d0304962868fbdd80da6a323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d407868d0304962868fbdd80da6a323" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43301 } } }
I20260812 06:56:07.432025 32702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:07.432798 32703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d407868d0304962868fbdd80da6a323. Latest consensus state: current_term: 1 leader_uuid: "9d407868d0304962868fbdd80da6a323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d407868d0304962868fbdd80da6a323" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43301 } } }
I20260812 06:56:07.433219 32703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:07.436694 32712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:07.442828 32712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:07.452389 32712 catalog_manager.cc:1347] Generated new cluster ID: 8ab4a00bc010481cb32371624b450dbd
I20260812 06:56:07.452534 32712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:07.461432 32712 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:07.462806 32712 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:07.477051 32712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d407868d0304962868fbdd80da6a323: Generated new TSK 0
I20260812 06:56:07.477975 32712 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:56:07.549976 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43301
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:07.904851 32734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:07.905153 32734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:07.905251 32734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:07.912822 32734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:07.913018 32734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:07.913160 32734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:07.923062 32734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:07.924474 32734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:07.925861 32734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:07.939633 32747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:07.939687 32744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:07.939913 32743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:07.940816 32734 server_base.cc:1034] running on GCE node
I20260812 06:56:07.941417 32734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:07.942317 32734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:07.943539 32734 hybrid_clock.cc:648] HybridClock initialized: now 1786517767943492 us; error 26 us; skew 500 ppm
I20260812 06:56:07.943895 32734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:07 dist-test-slave-t3q3 krb5kdc[32570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517767, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:07.950336 32734 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:56:07.951794 32734 webserver.cc:460] Webserver started at http://127.26.226.1:40669/ using document root <none> and password file <none>
I20260812 06:56:07.952535 32734 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:07.952648 32734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:07.952924 32734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:07.955996 32734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "2baec43e6ec045e596f8cec189b25201"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-t3q3"
server_key: "3cd026448c2413507af8c7ab4e368ead"
server_key_iv: "c02a45baddc0df8441a465149e12f163"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.956845 32734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "2baec43e6ec045e596f8cec189b25201"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-t3q3"
server_key: "3cd026448c2413507af8c7ab4e368ead"
server_key_iv: "c02a45baddc0df8441a465149e12f163"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.962225 32734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:56:07.965900 32758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:07.967355 32734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:07.967548 32734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "2baec43e6ec045e596f8cec189b25201"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-t3q3"
server_key: "3cd026448c2413507af8c7ab4e368ead"
server_key_iv: "c02a45baddc0df8441a465149e12f163"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.967772 32734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:07.996654 32734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:08.001003 32734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:08.001346 32734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:08.002393 32734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:08.003878 32734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:08.003989 32734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:08.004175 32734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:08.004261 32734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:08.039825 32734 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:40935
I20260812 06:56:08.039873 475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:40935 every 8 connection(s)
I20260812 06:56:08.041234 32734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:08.045791 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 32734
I20260812 06:56:08.046041 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:08.046983 27528 external_mini_cluster.cc:1428] Setting key 16fa0c6ea60e397a50d2ed81641ca487
Aug 12 06:56:08 dist-test-slave-t3q3 krb5kdc[32570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517767, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:08.064630 478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.043972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:45787 (local address 127.26.226.62:43301)
0812 06:56:08.044450 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.044455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.045548 (+ 1093us) server_negotiation.cc:407] Connection header received
0812 06:56:08.048196 (+ 2648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.048200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.048267 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.048362 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:08.050443 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.051178 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.053773 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.054173 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.054288 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.058668 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:08.058673 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:08.058677 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:08.058717 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:08.061424 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.062188 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.062193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.062196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.062322 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.062779 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.062791 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.062793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.063054 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:08.063217 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.063938 (+ 721us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.064232 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":258,"threads_started":1}
I20260812 06:56:08.065757 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.044547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43301 (local address 127.26.226.1:45787)
0812 06:56:08.045321 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:08.045397 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:08.047867 (+ 2470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:08.048659 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:08.048671 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:08.049379 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:08.050248 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.050277 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.051375 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.051384 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.053615 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.053635 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.054436 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.054447 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.054795 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.055572 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:08.055608 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:08.058422 (+ 2814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:08.061576 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:08.061586 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:08.061604 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:08.062043 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:08.062440 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:08.062446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:08.062449 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:08.062649 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:08.063202 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:08.063210 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:08.063550 (+ 340us) client_negotiation.cc:769] Sending connection context
0812 06:56:08.063828 (+ 278us) client_negotiation.cc:240] Negotiation successful
0812 06:56:08.064441 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":289,"threads_started":1}
I20260812 06:56:08.067477 476 heartbeater.cc:347] Connected to a master server at 127.26.226.62:43301
I20260812 06:56:08.067864 476 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:08.068953 476 heartbeater.cc:511] Master 127.26.226.62:43301 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
I20260812 06:56:08.071450 32643 ts_manager.cc:194] Registered new tserver with Master: 2baec43e6ec045e596f8cec189b25201 (127.26.226.1:40935)
I20260812 06:56:08.073220 32643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:45787
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:56:08.103507 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43301
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:08.456284 489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:08.456611 489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:08.456826 489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:08.464102 489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:08.464231 489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:08.464447 489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:08.474658 489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:08.476022 489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:08.477367 489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:08.490536 498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:08.490626 500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:08.490895 497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:08.491510 489 server_base.cc:1034] running on GCE node
I20260812 06:56:08.492205 489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:08.493137 489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:08.494346 489 hybrid_clock.cc:648] HybridClock initialized: now 1786517768494303 us; error 32 us; skew 500 ppm
I20260812 06:56:08.494679 489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:08 dist-test-slave-t3q3 krb5kdc[32570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517768, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:08.498054 489 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:56:08.499514 489 webserver.cc:460] Webserver started at http://127.26.226.2:36387/ using document root <none> and password file <none>
I20260812 06:56:08.500286 489 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:08.500439 489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:08.500757 489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:08.503814 489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "1feb1f65bbeb4d51b9ea1489620486e4"
format_stamp: "Formatted at 2026-08-12 06:56:08 on dist-test-slave-t3q3"
server_key: "eab102edb6439acf6449f831abe8339e"
server_key_iv: "42e2ec09e4e8bb28071600f2285cc901"
server_key_version: "encryptionkey@0"
I20260812 06:56:08.504719 489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "1feb1f65bbeb4d51b9ea1489620486e4"
format_stamp: "Formatted at 2026-08-12 06:56:08 on dist-test-slave-t3q3"
server_key: "eab102edb6439acf6449f831abe8339e"
server_key_iv: "42e2ec09e4e8bb28071600f2285cc901"
server_key_version: "encryptionkey@0"
I20260812 06:56:08.510006 489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 06:56:08.513687 511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:08.515035 489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:08.515225 489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "1feb1f65bbeb4d51b9ea1489620486e4"
format_stamp: "Formatted at 2026-08-12 06:56:08 on dist-test-slave-t3q3"
server_key: "eab102edb6439acf6449f831abe8339e"
server_key_iv: "42e2ec09e4e8bb28071600f2285cc901"
server_key_version: "encryptionkey@0"
I20260812 06:56:08.515442 489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:08.537667 489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:08.542397 489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:08.542737 489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:08.543874 489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:08.545351 489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:08.545468 489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:08.545624 489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:08.545702 489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:08.580346 489 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:36639
I20260812 06:56:08.580387 690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:36639 every 8 connection(s)
I20260812 06:56:08.581709 489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:08.587353 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 489
I20260812 06:56:08.587543 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:08.588109 27528 external_mini_cluster.cc:1428] Setting key c09b28c79c69b0e54e63d21b81c219b4
Aug 12 06:56:08 dist-test-slave-t3q3 krb5kdc[32570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517768, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:08.603533 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.584535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:48195 (local address 127.26.226.62:43301)
0812 06:56:08.584998 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.585003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.586009 (+ 1006us) server_negotiation.cc:407] Connection header received
0812 06:56:08.587965 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.587971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.588088 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.588227 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:08.589987 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.590785 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.592448 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.592895 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.593054 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.596905 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:08.596910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:08.596913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:08.596945 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:08.599682 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.600765 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.600773 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.600778 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.601026 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.601639 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.601644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.601647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.601961 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:08.602131 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.602888 (+ 757us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.603131 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":239,"threads_started":1}
I20260812 06:56:08.604627 695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.585087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43301 (local address 127.26.226.2:48195)
0812 06:56:08.585835 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:08.585874 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:08.587678 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:08.588477 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:08.588486 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:08.589139 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:08.589782 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.589811 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.590946 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.590952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.592291 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.592304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.593152 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.593161 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.593400 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.594121 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:08.594144 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:08.596647 (+ 2503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:08.599849 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:08.599861 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:08.599881 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:08.600604 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:08.601220 (+ 616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:08.601228 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:08.601233 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:08.601497 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:08.602105 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:08.602114 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:08.602502 (+ 388us) client_negotiation.cc:769] Sending connection context
0812 06:56:08.602786 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 06:56:08.603374 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":287,"threads_started":1}
I20260812 06:56:08.606192 691 heartbeater.cc:347] Connected to a master server at 127.26.226.62:43301
I20260812 06:56:08.606521 691 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:08.607299 691 heartbeater.cc:511] Master 127.26.226.62:43301 requested a full tablet report, sending...
I20260812 06:56:08.608942 32643 ts_manager.cc:194] Registered new tserver with Master: 1feb1f65bbeb4d51b9ea1489620486e4 (127.26.226.2:36639)
I20260812 06:56:08.609795 32643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:48195
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:56:08.641505 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43301
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:08.985793 698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:08.986083 698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:08.986364 698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:08.993844 698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:08.993970 698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:08.994213 698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:09.004289 698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:09.005690 698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:09.007036 698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:09.021034 705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:09.021126 709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:09.021063 706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:09.021803 698 server_base.cc:1034] running on GCE node
I20260812 06:56:09.022408 698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:09.023361 698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:09.024602 698 hybrid_clock.cc:648] HybridClock initialized: now 1786517769024560 us; error 38 us; skew 500 ppm
I20260812 06:56:09.024946 698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:09 dist-test-slave-t3q3 krb5kdc[32570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517769, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:09.028229 698 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:56:09.029889 698 webserver.cc:460] Webserver started at http://127.26.226.3:44019/ using document root <none> and password file <none>
I20260812 06:56:09.030700 698 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:09.030858 698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:09.031282 698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:09.034391 698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "fb76677199c94b2ea7a20ea8094522f8"
format_stamp: "Formatted at 2026-08-12 06:56:09 on dist-test-slave-t3q3"
server_key: "77de5fece716c1eff47c35b8f799e9fd"
server_key_iv: "4fd0698a483c6d7e63a6e677651c7150"
server_key_version: "encryptionkey@0"
I20260812 06:56:09.035224 698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "fb76677199c94b2ea7a20ea8094522f8"
format_stamp: "Formatted at 2026-08-12 06:56:09 on dist-test-slave-t3q3"
server_key: "77de5fece716c1eff47c35b8f799e9fd"
server_key_iv: "4fd0698a483c6d7e63a6e677651c7150"
server_key_version: "encryptionkey@0"
I20260812 06:56:09.040913 698 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 06:56:09.044651 721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:09.046130 698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 06:56:09.046283 698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "fb76677199c94b2ea7a20ea8094522f8"
format_stamp: "Formatted at 2026-08-12 06:56:09 on dist-test-slave-t3q3"
server_key: "77de5fece716c1eff47c35b8f799e9fd"
server_key_iv: "4fd0698a483c6d7e63a6e677651c7150"
server_key_version: "encryptionkey@0"
I20260812 06:56:09.046504 698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:09.065004 698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:09.068926 698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:09.069195 698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:09.070111 698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:09.071482 698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:09.071558 698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:09.071657 698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:09.071705 698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:09.076474 476 heartbeater.cc:503] Master 127.26.226.62:43301 was elected leader, sending a full tablet report...
I20260812 06:56:09.106949 698 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:39509
I20260812 06:56:09.107000 904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:39509 every 8 connection(s)
I20260812 06:56:09.108415 698 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:09.115316 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 698
I20260812 06:56:09.115530 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:09.116001 27528 external_mini_cluster.cc:1428] Setting key 5df475c6cd3cebc5de561f92ddb3c3d7
Aug 12 06:56:09 dist-test-slave-t3q3 krb5kdc[32570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517769, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:09.129428 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:09.111013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:44081 (local address 127.26.226.62:43301)
0812 06:56:09.111507 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:09.111514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:09.112561 (+ 1047us) server_negotiation.cc:407] Connection header received
0812 06:56:09.114323 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:09.114328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:09.114397 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:09.114499 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:09.116317 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.117241 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.119418 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.119963 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.120123 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:09.123567 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:09.123572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:09.123575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:09.123609 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:09.126282 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.127021 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.127026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.127028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.127155 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.127546 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.127549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.127551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.127834 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:09.127957 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:09.128791 (+ 834us) server_negotiation.cc:299] Negotiation successful
0812 06:56:09.129032 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":220,"threads_started":1}
I20260812 06:56:09.130666 911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:09.111612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43301 (local address 127.26.226.3:44081)
0812 06:56:09.112393 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:09.112445 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:09.114054 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:09.114709 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:09.114717 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:09.115405 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:09.116103 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:09.116138 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.117415 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.117419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:09.119276 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:09.119292 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.120231 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.120236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:09.120437 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:09.121086 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:09.121114 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:09.123367 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:09.126426 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:09.126435 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:09.126450 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:09.126905 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:09.127252 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:09.127256 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:09.127258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:09.127443 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:09.127946 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:09.127952 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:09.128391 (+ 439us) client_negotiation.cc:769] Sending connection context
0812 06:56:09.128685 (+ 294us) client_negotiation.cc:240] Negotiation successful
0812 06:56:09.129227 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":283,"threads_started":1}
I20260812 06:56:09.132359 905 heartbeater.cc:347] Connected to a master server at 127.26.226.62:43301
I20260812 06:56:09.132678 905 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:09.133610 905 heartbeater.cc:511] Master 127.26.226.62:43301 requested a full tablet report, sending...
I20260812 06:56:09.135150 32643 ts_manager.cc:194] Registered new tserver with Master: fb76677199c94b2ea7a20ea8094522f8 (127.26.226.3:39509)
I20260812 06:56:09.136283 32643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:44081
I20260812 06:56:09.144726 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:56:09 dist-test-slave-t3q3 krb5kdc[32570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517769, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 06:56:09 dist-test-slave-t3q3 krb5kdc[32570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517769, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:09.203372 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:09.188627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60384 (local address 127.26.226.62:43301)
0812 06:56:09.188839 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:09.188847 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:09.189377 (+ 530us) server_negotiation.cc:407] Connection header received
0812 06:56:09.189607 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:09.189612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:09.189681 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:09.189800 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:09.191274 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.192031 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.193334 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.193768 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.193916 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:09.197926 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:09.197932 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:09.197935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:09.197970 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:09.200572 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.201262 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.201268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.201273 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.201420 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.201837 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.201842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.201846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.202192 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:09.202466 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:09.202802 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 06:56:09.203035 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 06:56:09.204049 32643 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60384
I20260812 06:56:09.206177 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32734
Aug 12 06:56:09 dist-test-slave-t3q3 krb5kdc[32570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517769, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:56:09.222545 936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:09.207321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.26.226.1:40935)
0812 06:56:09.208013 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:09.208020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:09.208045 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:56:09.208159 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:09.208165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:09.208392 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:09.208521 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:09.209664 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.210674 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.211828 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.212461 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.212563 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:09.216831 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:09.216844 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:09.216858 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:09.216897 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:09.219773 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.220570 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.220577 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.220579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.220722 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.221083 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.221087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.221089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.221435 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:09.221665 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:09.221927 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 06:56:09.222129 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":368,"threads_started":1}
W20260812 06:56:09.223845 470 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59526
W20260812 06:56:09.225170 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32734: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:09.225425 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 32734
I20260812 06:56:09.253352 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 489
Aug 12 06:56:09 dist-test-slave-t3q3 krb5kdc[32570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517769, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:56:09.270320 937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:09.254435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.26.226.2:36639)
0812 06:56:09.255037 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:09.255044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:09.255071 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:56:09.255135 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:09.255141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:09.255364 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:09.255498 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:09.256964 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.258042 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.259332 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.259887 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.259991 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:09.264561 (+ 4570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:09.264574 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:09.264589 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:09.264632 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:09.267642 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.268374 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.268382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.268384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.268532 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.268852 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.268856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.268858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.269210 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:09.269443 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:09.269695 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 06:56:09.269950 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":332,"threads_started":1}
W20260812 06:56:09.271644 685 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38256
W20260812 06:56:09.272862 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 489: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:09.273041 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 489
I20260812 06:56:09.304565 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 698
Aug 12 06:56:09 dist-test-slave-t3q3 krb5kdc[32570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517769, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:56:09.321878 938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:09.305741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35974 (local address 127.26.226.3:39509)
0812 06:56:09.306532 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:09.306539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:09.306568 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:56:09.306634 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:09.306640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:09.306868 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:09.307006 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:09.308449 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.309516 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.310828 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.311390 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.311491 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:09.316252 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:09.316266 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:09.316282 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:09.316320 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:09.319327 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.320002 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.320009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.320011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.320202 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.320536 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.320539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.320541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.320889 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:09.321118 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:09.321382 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 06:56:09.321589 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":460,"threads_started":1}
W20260812 06:56:09.323212 899 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35974
W20260812 06:56:09.324332 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 698: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:09.324525 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 698
I20260812 06:56:09.365286 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32591
I20260812 06:56:09.366612 32686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:09.472913 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 32591
2026-08-12T06:56:09Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4118 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:09 dist-test-slave-t3q3 krb5kdc[945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:09 dist-test-slave-t3q3 krb5kdc[945](info): set up 2 sockets
Aug 12 06:56:09 dist-test-slave-t3q3 krb5kdc[945](info): commencing operation
krb5kdc: starting...
W20260812 06:56:10.833400 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.009s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:56:10 dist-test-slave-t3q3 krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:56:11.080744 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:39127
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:39127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:11.431504 963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:11.431790 963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:11.431946 963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:11.439324 963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:11.439466 963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:11.439548 963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:11.439641 963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:11.439703 963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:11.449836 963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:39127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:39127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:11.451260 963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:11.452705 963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:11.463917 974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:11.464201 973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:11.465799 978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:11.466032 963 server_base.cc:1034] running on GCE node
I20260812 06:56:11.466876 963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:11.468441 963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:11.469726 963 hybrid_clock.cc:648] HybridClock initialized: now 1786517771469650 us; error 55 us; skew 500 ppm
I20260812 06:56:11.470165 963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:11 dist-test-slave-t3q3 krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517771, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:11.473472 963 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:56:11.474993 963 webserver.cc:460] Webserver started at http://127.26.226.62:41911/ using document root <none> and password file <none>
I20260812 06:56:11.475728 963 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:11.475884 963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:11.476315 963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:11.479343 963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "b2df75ebcb284449a00e1f76d28822b2"
format_stamp: "Formatted at 2026-08-12 06:56:11 on dist-test-slave-t3q3"
server_key: "efbe3b5458ed00cbae6f22282d0f496a"
server_key_iv: "fa52cd0123e65e843ec19175fd87cfd4"
server_key_version: "encryptionkey@0"
I20260812 06:56:11.480377 963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "b2df75ebcb284449a00e1f76d28822b2"
format_stamp: "Formatted at 2026-08-12 06:56:11 on dist-test-slave-t3q3"
server_key: "efbe3b5458ed00cbae6f22282d0f496a"
server_key_iv: "fa52cd0123e65e843ec19175fd87cfd4"
server_key_version: "encryptionkey@0"
I20260812 06:56:11.485632 963 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:56:11.489266 989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:11.490507 963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:11.490705 963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "b2df75ebcb284449a00e1f76d28822b2"
format_stamp: "Formatted at 2026-08-12 06:56:11 on dist-test-slave-t3q3"
server_key: "efbe3b5458ed00cbae6f22282d0f496a"
server_key_iv: "fa52cd0123e65e843ec19175fd87cfd4"
server_key_version: "encryptionkey@0"
I20260812 06:56:11.490914 963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:11.502715 963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:11.506630 963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:11.506919 963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:11.530377 963 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:39127
I20260812 06:56:11.530400 1082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:39127 every 8 connection(s)
I20260812 06:56:11.531868 963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:11.535627 1083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:11.536279 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 963
I20260812 06:56:11.536528 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:11.537220 27528 external_mini_cluster.cc:1428] Setting key c594117e72c72ae18445080207256340
I20260812 06:56:11.549835 1083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2: Bootstrap starting.
Aug 12 06:56:11 dist-test-slave-t3q3 krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:11.553565 1083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:11.554697 1083 log.cc:824] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:11.558033 1083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2: No bootstrap required, opened a new log
I20260812 06:56:11.561714 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.539362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47898 (local address 127.26.226.62:39127)
0812 06:56:11.540232 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.540244 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.540288 (+ 44us) server_negotiation.cc:407] Connection header received
0812 06:56:11.541932 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.541956 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.542341 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.542672 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:11.544554 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.545813 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.547644 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.548518 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.548657 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.553378 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:11.553386 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:11.553401 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:11.553436 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:11.557180 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:11.557842 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:11.557850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:11.557860 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:11.558069 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:11.558507 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:11.558517 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:11.558521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:11.559231 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:11.559531 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.559886 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.560384 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":396,"threads_started":1}
I20260812 06:56:11.566751 1083 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df75ebcb284449a00e1f76d28822b2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39127 } }
I20260812 06:56:11.567073 1083 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:11.567152 1083 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2df75ebcb284449a00e1f76d28822b2, State: Initialized, Role: FOLLOWER
I20260812 06:56:11.567620 1083 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df75ebcb284449a00e1f76d28822b2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39127 } }
I20260812 06:56:11.567821 1083 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:11.567952 1083 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:11.568168 1083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:11.570552 1083 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df75ebcb284449a00e1f76d28822b2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39127 } }
I20260812 06:56:11.571022 1083 leader_election.cc:302] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2df75ebcb284449a00e1f76d28822b2; no voters:
I20260812 06:56:11.571627 1083 leader_election.cc:288] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:11.571784 1090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:11.573077 1090 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [term 1 LEADER]: Becoming Leader. State: Replica: b2df75ebcb284449a00e1f76d28822b2, State: Running, Role: LEADER
I20260812 06:56:11.573673 1090 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df75ebcb284449a00e1f76d28822b2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39127 } }
I20260812 06:56:11.573880 1083 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:11.576694 1093 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2df75ebcb284449a00e1f76d28822b2. Latest consensus state: current_term: 1 leader_uuid: "b2df75ebcb284449a00e1f76d28822b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df75ebcb284449a00e1f76d28822b2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39127 } } }
I20260812 06:56:11.577088 1093 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:11.577639 1092 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2df75ebcb284449a00e1f76d28822b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df75ebcb284449a00e1f76d28822b2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39127 } } }
I20260812 06:56:11.577972 1092 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:11.579370 1103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:11.584765 1103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:11.593281 1103 catalog_manager.cc:1347] Generated new cluster ID: 06515f09398c492783aa80aa543dd7fe
I20260812 06:56:11.593396 1103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:11.609107 1103 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:11.610540 1103 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:11.621768 1103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2df75ebcb284449a00e1f76d28822b2: Generated new TSK 0
I20260812 06:56:11.622685 1103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:56:11.686429 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39127
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:12.033555 1124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:12.033917 1124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:12.034101 1124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:12.041697 1124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:12.041860 1124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:12.042052 1124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:12.052152 1124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:12.053572 1124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:12.055011 1124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:12.068958 1133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:12.069020 1131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:12.069298 1136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:12.070099 1124 server_base.cc:1034] running on GCE node
I20260812 06:56:12.070772 1124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:12.071849 1124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:12.073114 1124 hybrid_clock.cc:648] HybridClock initialized: now 1786517772073066 us; error 38 us; skew 500 ppm
I20260812 06:56:12.073489 1124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:12 dist-test-slave-t3q3 krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517772, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:12.077144 1124 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:56:12.078816 1124 webserver.cc:460] Webserver started at http://127.26.226.1:40931/ using document root <none> and password file <none>
I20260812 06:56:12.079546 1124 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:12.079704 1124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:12.080046 1124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:12.083153 1124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "597e07a7f92c46fc81240400459dcddf"
format_stamp: "Formatted at 2026-08-12 06:56:12 on dist-test-slave-t3q3"
server_key: "085948b37bc89b6dc06f08b5a1be8ac6"
server_key_iv: "b9c80e253ec9bf7312a0773e2a0acfcf"
server_key_version: "encryptionkey@0"
I20260812 06:56:12.084038 1124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "597e07a7f92c46fc81240400459dcddf"
format_stamp: "Formatted at 2026-08-12 06:56:12 on dist-test-slave-t3q3"
server_key: "085948b37bc89b6dc06f08b5a1be8ac6"
server_key_iv: "b9c80e253ec9bf7312a0773e2a0acfcf"
server_key_version: "encryptionkey@0"
I20260812 06:56:12.089742 1124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260812 06:56:12.093510 1148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:12.094956 1124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:12.095144 1124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "597e07a7f92c46fc81240400459dcddf"
format_stamp: "Formatted at 2026-08-12 06:56:12 on dist-test-slave-t3q3"
server_key: "085948b37bc89b6dc06f08b5a1be8ac6"
server_key_iv: "b9c80e253ec9bf7312a0773e2a0acfcf"
server_key_version: "encryptionkey@0"
I20260812 06:56:12.095400 1124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:12.114753 1124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:12.118891 1124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:12.119151 1124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:12.120177 1124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:12.121956 1124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:12.122049 1124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:12.122154 1124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:12.122205 1124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:12.158175 1124 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:33989
I20260812 06:56:12.158234 1333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:33989 every 8 connection(s)
I20260812 06:56:12.159555 1124 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:12.163175 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 1124
I20260812 06:56:12.163424 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:12.164345 27528 external_mini_cluster.cc:1428] Setting key 2273629951e2b147ea45229f8b94a0ec
Aug 12 06:56:12 dist-test-slave-t3q3 krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517772, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:12.183146 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:12.162620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:36025 (local address 127.26.226.62:39127)
0812 06:56:12.163138 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:12.163143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:12.164288 (+ 1145us) server_negotiation.cc:407] Connection header received
0812 06:56:12.166371 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:12.166374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:12.166486 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:12.166608 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:12.168124 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:12.169324 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:12.170856 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:12.171574 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:12.171738 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:12.175861 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:12.175871 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:12.175878 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:12.175924 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:12.179642 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:12.180466 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:12.180475 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:12.180482 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:12.180639 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:12.181064 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:12.181072 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:12.181079 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:12.181479 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:12.181651 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:12.182381 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 06:56:12.182652 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":265,"threads_started":1}
I20260812 06:56:12.183804 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:12.163239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:39127 (local address 127.26.226.1:36025)
0812 06:56:12.164104 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:12.164165 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:12.166060 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:12.166820 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:12.166828 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:12.167358 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:12.167917 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:12.167934 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:12.169476 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:12.169480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:12.170706 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:12.170718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:12.171800 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:12.171805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:12.171997 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:12.172763 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:12.172793 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:12.175626 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:12.179785 (+ 4159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:12.179793 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:12.179807 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:12.180339 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:12.180756 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:12.180759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:12.180762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:12.180942 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:12.181610 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:12.181616 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:12.181957 (+ 341us) client_negotiation.cc:769] Sending connection context
0812 06:56:12.182194 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 06:56:12.182612 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":339,"threads_started":1}
I20260812 06:56:12.185520 1335 heartbeater.cc:347] Connected to a master server at 127.26.226.62:39127
I20260812 06:56:12.185873 1335 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:12.186832 1335 heartbeater.cc:511] Master 127.26.226.62:39127 requested a full tablet report, sending...
I20260812 06:56:12.189231 1020 ts_manager.cc:194] Registered new tserver with Master: 597e07a7f92c46fc81240400459dcddf (127.26.226.1:33989)
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
I20260812 06:56:12.190706 1020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:36025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:56:12.222646 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39127
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:12.576526 1343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:12.576810 1343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:12.576908 1343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:12.585047 1343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:12.585168 1343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:12.585285 1343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:12.595571 1343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:12.596907 1343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:12.598157 1343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:12.611560 1352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:12.611680 1353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:12.611770 1358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:12.613031 1343 server_base.cc:1034] running on GCE node
I20260812 06:56:12.613672 1343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:12.614558 1343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:12.615770 1343 hybrid_clock.cc:648] HybridClock initialized: now 1786517772615722 us; error 36 us; skew 500 ppm
I20260812 06:56:12.616150 1343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:12 dist-test-slave-t3q3 krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517772, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:12.619355 1343 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:56:12.620820 1343 webserver.cc:460] Webserver started at http://127.26.226.2:42927/ using document root <none> and password file <none>
I20260812 06:56:12.621528 1343 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:12.621667 1343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:12.622097 1343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:12.625125 1343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "9ca4558304b544b9b813b306051e1c43"
format_stamp: "Formatted at 2026-08-12 06:56:12 on dist-test-slave-t3q3"
server_key: "54f1ef76ee6768df8d089dd2048ae00b"
server_key_iv: "6df7010b2e46b72779849952b1da85e7"
server_key_version: "encryptionkey@0"
I20260812 06:56:12.625881 1343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "9ca4558304b544b9b813b306051e1c43"
format_stamp: "Formatted at 2026-08-12 06:56:12 on dist-test-slave-t3q3"
server_key: "54f1ef76ee6768df8d089dd2048ae00b"
server_key_iv: "6df7010b2e46b72779849952b1da85e7"
server_key_version: "encryptionkey@0"
I20260812 06:56:12.630960 1343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:56:12.634343 1373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:12.635615 1343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:56:12.635800 1343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "9ca4558304b544b9b813b306051e1c43"
format_stamp: "Formatted at 2026-08-12 06:56:12 on dist-test-slave-t3q3"
server_key: "54f1ef76ee6768df8d089dd2048ae00b"
server_key_iv: "6df7010b2e46b72779849952b1da85e7"
server_key_version: "encryptionkey@0"
I20260812 06:56:12.636010 1343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:12.655179 1343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:12.660167 1343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:12.660483 1343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:12.661548 1343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:12.662981 1343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:12.663086 1343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:12.663242 1343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:12.663333 1343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:12.698614 1343 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:36393
I20260812 06:56:12.698688 1556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:36393 every 8 connection(s)
I20260812 06:56:12.700090 1343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:12.708106 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 1343
I20260812 06:56:12.708312 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:12.708837 27528 external_mini_cluster.cc:1428] Setting key 7edbc55cc44d42f5a722b7f82ea0ca21
Aug 12 06:56:12 dist-test-slave-t3q3 krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517772, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:12.721271 1560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:12.702790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:39613 (local address 127.26.226.62:39127)
0812 06:56:12.703274 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:12.703281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:12.704275 (+ 994us) server_negotiation.cc:407] Connection header received
0812 06:56:12.706024 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:12.706028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:12.706106 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:12.706209 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:12.707668 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:12.708800 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:12.710370 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:12.711005 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:12.711110 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:12.715441 (+ 4331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:12.715446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:12.715449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:12.715480 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:12.718079 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:12.718832 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:12.718840 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:12.718846 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:12.718981 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:12.719428 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:12.719434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:12.719437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:12.719708 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:12.719869 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:12.720649 (+ 780us) server_negotiation.cc:299] Negotiation successful
0812 06:56:12.720897 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":215,"threads_started":1}
I20260812 06:56:12.722222 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:12.703318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:39127 (local address 127.26.226.2:39613)
0812 06:56:12.704047 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:12.704159 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:12.705783 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:12.706403 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:12.706410 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:12.706942 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:12.707480 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:12.707497 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:12.709011 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:12.709015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:12.710226 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:12.710239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:12.711280 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:12.711288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:12.711586 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:12.712398 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:12.712442 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:12.715215 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:12.718241 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:12.718250 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:12.718266 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:12.718701 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:12.719097 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:12.719102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:12.719106 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:12.719306 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:12.719846 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:12.719854 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:12.720277 (+ 423us) client_negotiation.cc:769] Sending connection context
0812 06:56:12.720560 (+ 283us) client_negotiation.cc:240] Negotiation successful
0812 06:56:12.721091 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":281,"threads_started":1}
I20260812 06:56:12.723790 1558 heartbeater.cc:347] Connected to a master server at 127.26.226.62:39127
I20260812 06:56:12.724215 1558 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:12.724944 1558 heartbeater.cc:511] Master 127.26.226.62:39127 requested a full tablet report, sending...
I20260812 06:56:12.726537 1020 ts_manager.cc:194] Registered new tserver with Master: 9ca4558304b544b9b813b306051e1c43 (127.26.226.2:36393)
I20260812 06:56:12.727398 1020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:39613
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:56:12.765839 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39127
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:13.115984 1569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:13.116362 1569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:13.116464 1569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:13.123862 1569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:13.123979 1569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:13.124162 1569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:13.134465 1569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:13.135787 1569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:13.137288 1569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:13.152429 1578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:13.152462 1579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:13.152377 1581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:13.154323 1569 server_base.cc:1034] running on GCE node
I20260812 06:56:13.155016 1569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:13.156236 1569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:13.157536 1569 hybrid_clock.cc:648] HybridClock initialized: now 1786517773157480 us; error 57 us; skew 500 ppm
I20260812 06:56:13.157984 1569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:13 dist-test-slave-t3q3 krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:13.166539 1569 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:56:13.168515 1569 webserver.cc:460] Webserver started at http://127.26.226.3:39481/ using document root <none> and password file <none>
I20260812 06:56:13.169378 1569 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:13.169561 1569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:13.169945 1569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:13.173609 1569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "49c3522c38734a25a562948dd625eb3f"
format_stamp: "Formatted at 2026-08-12 06:56:13 on dist-test-slave-t3q3"
server_key: "c9fd34e412307a258c3e53ddc183d008"
server_key_iv: "e1f671f96e653a6a64f944bc6fd53507"
server_key_version: "encryptionkey@0"
I20260812 06:56:13.174654 1569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "49c3522c38734a25a562948dd625eb3f"
format_stamp: "Formatted at 2026-08-12 06:56:13 on dist-test-slave-t3q3"
server_key: "c9fd34e412307a258c3e53ddc183d008"
server_key_iv: "e1f671f96e653a6a64f944bc6fd53507"
server_key_version: "encryptionkey@0"
I20260812 06:56:13.181048 1569 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 06:56:13.185138 1591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:13.186574 1569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:13.186826 1569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "49c3522c38734a25a562948dd625eb3f"
format_stamp: "Formatted at 2026-08-12 06:56:13 on dist-test-slave-t3q3"
server_key: "c9fd34e412307a258c3e53ddc183d008"
server_key_iv: "e1f671f96e653a6a64f944bc6fd53507"
server_key_version: "encryptionkey@0"
I20260812 06:56:13.187083 1569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:13.193972 1335 heartbeater.cc:503] Master 127.26.226.62:39127 was elected leader, sending a full tablet report...
I20260812 06:56:13.199155 1569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:13.203944 1569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:13.204319 1569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:13.205333 1569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:13.206772 1569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:13.206851 1569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:13.207015 1569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:13.207067 1569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:13.241142 1569 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:46735
I20260812 06:56:13.241305 1775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:46735 every 8 connection(s)
I20260812 06:56:13.242690 1569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:13.252516 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 1569
I20260812 06:56:13.252736 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:13.253232 27528 external_mini_cluster.cc:1428] Setting key e3d71ece381a500fa61479f7eba9fa22
Aug 12 06:56:13 dist-test-slave-t3q3 krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:13.264236 1783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:13.245505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:40161 (local address 127.26.226.62:39127)
0812 06:56:13.246048 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:13.246054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:13.247079 (+ 1025us) server_negotiation.cc:407] Connection header received
0812 06:56:13.249032 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:13.249036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:13.249132 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:13.249240 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:13.250930 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.251780 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.253792 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.254239 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.254393 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:13.258235 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:13.258240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:13.258243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:13.258277 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:13.261067 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.261825 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.261829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.261832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.261982 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.262539 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.262545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.262548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.262874 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:13.263003 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:13.263697 (+ 694us) server_negotiation.cc:299] Negotiation successful
0812 06:56:13.263895 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":260,"threads_started":1}
I20260812 06:56:13.265228 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:13.246093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:39127 (local address 127.26.226.3:40161)
0812 06:56:13.246897 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:13.246938 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:13.248751 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:13.249476 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:13.249485 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:13.250074 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:13.250695 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:13.250714 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.252024 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.252029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:13.253624 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:13.253638 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.254496 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.254506 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:13.254781 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:13.255507 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:13.255530 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:13.258017 (+ 2487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:13.261207 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:13.261215 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:13.261228 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:13.261699 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:13.262102 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:13.262107 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:13.262110 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:13.262392 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:13.262996 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:13.263003 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:13.263357 (+ 354us) client_negotiation.cc:769] Sending connection context
0812 06:56:13.263604 (+ 247us) client_negotiation.cc:240] Negotiation successful
0812 06:56:13.264019 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":301,"threads_started":1}
I20260812 06:56:13.266691 1777 heartbeater.cc:347] Connected to a master server at 127.26.226.62:39127
I20260812 06:56:13.267022 1777 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:13.267771 1777 heartbeater.cc:511] Master 127.26.226.62:39127 requested a full tablet report, sending...
I20260812 06:56:13.269348 1020 ts_manager.cc:194] Registered new tserver with Master: 49c3522c38734a25a562948dd625eb3f (127.26.226.3:46735)
I20260812 06:56:13.270216 1020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:40161
I20260812 06:56:13.281272 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:56:13 dist-test-slave-t3q3 krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:56:13 dist-test-slave-t3q3 krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:56:13.342727 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:13.326479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56980 (local address 127.26.226.1:33989)
0812 06:56:13.327243 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:13.327254 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:13.327293 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:56:13.327378 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:13.327385 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:13.327607 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:13.327792 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:13.329435 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.330480 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.331769 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.332429 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.332577 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:13.336782 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:13.336795 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:13.336811 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:13.336849 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:13.339794 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.340637 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.340646 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.340649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.340782 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.341319 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.341323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.341325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.341688 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:13.341933 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:13.342197 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 06:56:13.342429 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":352,"threads_started":1}
Aug 12 06:56:13 dist-test-slave-t3q3 krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:13.361732 1783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:13.347462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.26.226.62:39127)
0812 06:56:13.347735 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:13.347743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:13.347958 (+ 215us) server_negotiation.cc:407] Connection header received
0812 06:56:13.348256 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:13.348267 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:13.348336 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:13.348445 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:13.349785 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.350551 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.351820 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.352300 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.352440 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:13.356523 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:13.356528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:13.356531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:13.356563 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:13.358958 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.359640 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.359652 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.359654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.359773 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.360246 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.360251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.360254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.360593 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:13.360860 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:13.361189 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 06:56:13.361421 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
W20260812 06:56:13.362339 1020 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47912
I20260812 06:56:13.363421 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 1124
I20260812 06:56:13.376185 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:13.364361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56996 (local address 127.26.226.1:33989)
0812 06:56:13.364598 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:13.364605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:13.364818 (+ 213us) server_negotiation.cc:407] Connection header received
0812 06:56:13.364936 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:13.364940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:13.365037 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:13.365120 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:13.366293 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.367004 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.368282 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.368701 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.368806 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:13.371015 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:13.371020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:13.371023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:13.371055 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:13.373646 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.374351 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.374355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.374358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.374494 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.374823 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.374826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.374828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.375124 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:13.375331 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:13.375595 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 06:56:13.375765 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:56:13.376806 1322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:13.460916 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 1124
I20260812 06:56:13.490561 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 1343
Aug 12 06:56:13 dist-test-slave-t3q3 krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:56:13.506876 1802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:13.491563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.26.226.2:36393)
0812 06:56:13.492284 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:13.492292 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:13.492318 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:56:13.492389 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:13.492395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:13.492601 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:13.492745 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:13.493944 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.494988 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.496447 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.496988 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.497089 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:13.501286 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:13.501298 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:13.501313 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:13.501350 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:13.504425 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.505002 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.505009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.505011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.505135 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.505448 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.505451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.505453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.505794 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:13.506017 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:13.506267 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 06:56:13.506469 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":381,"threads_started":1}
I20260812 06:56:13.508292 1549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:13.598773 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 1343
I20260812 06:56:13.627727 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 1569
Aug 12 06:56:13 dist-test-slave-t3q3 krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:56:13.644045 1804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:13.628991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.26.226.3:46735)
0812 06:56:13.629664 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:13.629671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:13.629703 (+ 32us) server_negotiation.cc:407] Connection header received
0812 06:56:13.629766 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:13.629772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:13.629981 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:13.630117 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:13.631381 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.632513 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.633789 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:13.634340 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:13.634440 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:13.638662 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:13.638675 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:13.638690 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:13.638724 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:13.641473 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.642193 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.642212 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.642215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.642338 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:13.642658 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:13.642662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:13.642664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:13.642998 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:13.643254 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:13.643440 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 06:56:13.643641 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":348,"threads_started":1}
I20260812 06:56:13.645440 1768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:13.731842 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 1569
I20260812 06:56:13.756690 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 963
I20260812 06:56:13.757963 1075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:13.864554 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 963
2026-08-12T06:56:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4392 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:13 dist-test-slave-t3q3 krb5kdc[1812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:13 dist-test-slave-t3q3 krb5kdc[1812](info): set up 2 sockets
Aug 12 06:56:13 dist-test-slave-t3q3 krb5kdc[1812](info): commencing operation
krb5kdc: starting...
W20260812 06:56:15.207324 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:56:15 dist-test-slave-t3q3 krb5kdc[1812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:56:15.479501 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:46661
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:46661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:15.818533 1828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:15.818830 1828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:15.819048 1828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:15.826184 1828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:15.826285 1828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:15.826397 1828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:15.826474 1828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:15.826540 1828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:15.836514 1828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:46661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:46661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:15.837844 1828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:15.839215 1828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:15.851418 1837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:15.851464 1835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:15.851630 1834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:15.852391 1828 server_base.cc:1034] running on GCE node
I20260812 06:56:15.853156 1828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:15.854450 1828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:15.855654 1828 hybrid_clock.cc:648] HybridClock initialized: now 1786517775855612 us; error 30 us; skew 500 ppm
I20260812 06:56:15.855957 1828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:15 dist-test-slave-t3q3 krb5kdc[1812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517775, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:15.859050 1828 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:56:15.860292 1828 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:15.860414 1828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:15.860759 1828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:15.863734 1828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "4fb07cbacbb44feeaea328ea2efb9c8f"
format_stamp: "Formatted at 2026-08-12 06:56:15 on dist-test-slave-t3q3"
server_key: "bc11c993cbdc5ea03b5983423c00129d"
server_key_iv: "e76b9b7270b1a294a103ca7c9a81f395"
server_key_version: "encryptionkey@0"
I20260812 06:56:15.864570 1828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "4fb07cbacbb44feeaea328ea2efb9c8f"
format_stamp: "Formatted at 2026-08-12 06:56:15 on dist-test-slave-t3q3"
server_key: "bc11c993cbdc5ea03b5983423c00129d"
server_key_iv: "e76b9b7270b1a294a103ca7c9a81f395"
server_key_version: "encryptionkey@0"
I20260812 06:56:15.869454 1828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:56:15.872915 1846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:15.874171 1828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:15.874354 1828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "4fb07cbacbb44feeaea328ea2efb9c8f"
format_stamp: "Formatted at 2026-08-12 06:56:15 on dist-test-slave-t3q3"
server_key: "bc11c993cbdc5ea03b5983423c00129d"
server_key_iv: "e76b9b7270b1a294a103ca7c9a81f395"
server_key_version: "encryptionkey@0"
I20260812 06:56:15.874591 1828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:15.902807 1828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:15.907433 1828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:15.907742 1828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:15.929705 1828 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:46661
I20260812 06:56:15.929678 1924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:46661 every 8 connection(s)
I20260812 06:56:15.930738 1828 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:15.933233 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 1828
I20260812 06:56:15.933478 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:15.934155 27528 external_mini_cluster.cc:1428] Setting key 963be3b9e1f6748a1173a968162a38b7
I20260812 06:56:15.935128 1926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 06:56:15 dist-test-slave-t3q3 krb5kdc[1812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:15.951473 1926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f: Bootstrap starting.
I20260812 06:56:15.955116 1926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:15.956423 1926 log.cc:824] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:15.959203 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.936323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59930 (local address 127.26.226.62:46661)
0812 06:56:15.937136 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.937148 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.937190 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:56:15.938745 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.938766 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.939152 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.939481 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:15.941762 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.943159 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.945196 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.945880 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.946021 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.951302 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:15.951311 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:15.951327 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:15.951365 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:15.954544 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.955272 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.955279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.955285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.955493 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.955912 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.955916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.955918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.956617 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:15.956941 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.957426 (+ 485us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.957909 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":387,"threads_started":1}
I20260812 06:56:15.959888 1926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f: No bootstrap required, opened a new log
I20260812 06:56:15.967571 1926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb07cbacbb44feeaea328ea2efb9c8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46661 } }
I20260812 06:56:15.967903 1926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:15.967979 1926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fb07cbacbb44feeaea328ea2efb9c8f, State: Initialized, Role: FOLLOWER
I20260812 06:56:15.968588 1926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb07cbacbb44feeaea328ea2efb9c8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46661 } }
I20260812 06:56:15.968875 1926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:15.969034 1926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:15.969228 1926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:15.971431 1926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb07cbacbb44feeaea328ea2efb9c8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46661 } }
I20260812 06:56:15.971920 1926 leader_election.cc:302] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fb07cbacbb44feeaea328ea2efb9c8f; no voters:
I20260812 06:56:15.972535 1926 leader_election.cc:288] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:15.972728 1933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:15.973953 1933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [term 1 LEADER]: Becoming Leader. State: Replica: 4fb07cbacbb44feeaea328ea2efb9c8f, State: Running, Role: LEADER
I20260812 06:56:15.974458 1933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb07cbacbb44feeaea328ea2efb9c8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46661 } }
I20260812 06:56:15.974694 1926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:15.977782 1936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fb07cbacbb44feeaea328ea2efb9c8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb07cbacbb44feeaea328ea2efb9c8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46661 } } }
I20260812 06:56:15.978253 1936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:15.979101 1937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fb07cbacbb44feeaea328ea2efb9c8f. Latest consensus state: current_term: 1 leader_uuid: "4fb07cbacbb44feeaea328ea2efb9c8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb07cbacbb44feeaea328ea2efb9c8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46661 } } }
I20260812 06:56:15.979512 1937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:15.983822 1946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:15.988168 1946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:15.996296 1946 catalog_manager.cc:1347] Generated new cluster ID: 43de12ab32b24e0087eda7725f8c2563
I20260812 06:56:15.996421 1946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:16.013357 1946 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:16.014788 1946 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:16.024567 1946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f: Generated new TSK 0
I20260812 06:56:16.025497 1946 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:56:16.087915 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46661
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:16.445310 1968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:16.445604 1968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:16.445782 1968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:16.453456 1968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:16.453581 1968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:16.453794 1968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:16.464449 1968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:16.465902 1968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:16.467345 1968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:16.481217 1977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:16.481197 1979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:16.481477 1984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:16.482179 1968 server_base.cc:1034] running on GCE node
I20260812 06:56:16.482875 1968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:16.483935 1968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:16.485183 1968 hybrid_clock.cc:648] HybridClock initialized: now 1786517776485129 us; error 68 us; skew 500 ppm
I20260812 06:56:16.485532 1968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:16 dist-test-slave-t3q3 krb5kdc[1812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517776, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:16.488941 1968 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:56:16.490118 1968 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:16.490214 1968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:16.490480 1968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:16.493615 1968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "5a3194b2e374479f9192145ad0d4b78a"
format_stamp: "Formatted at 2026-08-12 06:56:16 on dist-test-slave-t3q3"
server_key: "9abb42438c14ec6e2fc3b7c6a62899db"
server_key_iv: "eaaa88344881f5fba73200617ffe3df2"
server_key_version: "encryptionkey@0"
I20260812 06:56:16.494443 1968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "5a3194b2e374479f9192145ad0d4b78a"
format_stamp: "Formatted at 2026-08-12 06:56:16 on dist-test-slave-t3q3"
server_key: "9abb42438c14ec6e2fc3b7c6a62899db"
server_key_iv: "eaaa88344881f5fba73200617ffe3df2"
server_key_version: "encryptionkey@0"
I20260812 06:56:16.499645 1968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:56:16.503190 1994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:16.504590 1968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:16.504774 1968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "5a3194b2e374479f9192145ad0d4b78a"
format_stamp: "Formatted at 2026-08-12 06:56:16 on dist-test-slave-t3q3"
server_key: "9abb42438c14ec6e2fc3b7c6a62899db"
server_key_iv: "eaaa88344881f5fba73200617ffe3df2"
server_key_version: "encryptionkey@0"
I20260812 06:56:16.505012 1968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:16.531553 1968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:16.538805 1968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:16.539057 1968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:16.540197 1968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:16.541594 1968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:16.541687 1968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:16.541769 1968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:16.541826 1968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:16.577296 1968 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:45087
I20260812 06:56:16.577417 2173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:45087 every 8 connection(s)
I20260812 06:56:16.578296 1968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:16.587074 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 1968
I20260812 06:56:16.587311 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:16.588157 27528 external_mini_cluster.cc:1428] Setting key b0916869a63ec64405e99dec8c02b3f1
Aug 12 06:56:16 dist-test-slave-t3q3 krb5kdc[1812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517776, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:16.599797 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:16.580940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:53649 (local address 127.26.226.62:46661)
0812 06:56:16.581469 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:16.581477 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:16.582418 (+ 941us) server_negotiation.cc:407] Connection header received
0812 06:56:16.584469 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:16.584473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:16.584548 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:16.584655 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:16.586092 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:16.586802 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:16.589159 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:16.589751 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:16.589895 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:16.593999 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:16.594004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:16.594008 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:16.594040 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:16.596748 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:16.597493 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:16.597498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:16.597500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:16.597621 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:16.598070 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:16.598074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:16.598076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:16.598338 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:16.598501 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:16.599198 (+ 697us) server_negotiation.cc:299] Negotiation successful
0812 06:56:16.599433 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":244,"threads_started":1}
I20260812 06:56:16.600821 2180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:16.581513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:46661 (local address 127.26.226.1:53649)
0812 06:56:16.582244 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:16.582291 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:16.584223 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:16.584852 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:16.584859 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:16.585401 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:16.585943 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:16.585959 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:16.586964 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:16.586970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:16.588999 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:16.589015 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:16.590008 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:16.590017 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:16.590305 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:16.591002 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:16.591030 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:16.593763 (+ 2733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:16.596898 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:16.596907 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:16.596922 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:16.597354 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:16.597745 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:16.597750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:16.597754 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:16.597932 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:16.598475 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:16.598483 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:16.598822 (+ 339us) client_negotiation.cc:769] Sending connection context
0812 06:56:16.599080 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 06:56:16.599596 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":288,"threads_started":1}
I20260812 06:56:16.602253 2174 heartbeater.cc:347] Connected to a master server at 127.26.226.62:46661
I20260812 06:56:16.602576 2174 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:16.603399 2174 heartbeater.cc:511] Master 127.26.226.62:46661 requested a full tablet report, sending...
I20260812 06:56:16.605602 1872 ts_manager.cc:194] Registered new tserver with Master: 5a3194b2e374479f9192145ad0d4b78a (127.26.226.1:45087)
I20260812 06:56:16.607021 1872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:53649
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:56:16.643595 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46661
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:16.993360 2185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:16.993707 2185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:16.993892 2185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:17.001220 2185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:17.001379 2185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:17.001580 2185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:17.011559 2185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:17.013023 2185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:17.014392 2185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:17.027917 2191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:17.027947 2193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:17.028023 2190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:17.033806 2185 server_base.cc:1034] running on GCE node
I20260812 06:56:17.034577 2185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:17.035656 2185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:17.036897 2185 hybrid_clock.cc:648] HybridClock initialized: now 1786517777036847 us; error 38 us; skew 500 ppm
I20260812 06:56:17.037285 2185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:17 dist-test-slave-t3q3 krb5kdc[1812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517777, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:17.040760 2185 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:56:17.042075 2185 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:17.042214 2185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:17.042572 2185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:17.045748 2185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "cbce5397c862406fabaa9363928a627c"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-t3q3"
server_key: "4c5483deb474b49534bbdc8a711814d4"
server_key_iv: "e804c309a0eb4189c3945630f6ded1ce"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.046648 2185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "cbce5397c862406fabaa9363928a627c"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-t3q3"
server_key: "4c5483deb474b49534bbdc8a711814d4"
server_key_iv: "e804c309a0eb4189c3945630f6ded1ce"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.052615 2185 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260812 06:56:17.056318 2202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.057853 2185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:17.058058 2185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "cbce5397c862406fabaa9363928a627c"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-t3q3"
server_key: "4c5483deb474b49534bbdc8a711814d4"
server_key_iv: "e804c309a0eb4189c3945630f6ded1ce"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.058308 2185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:17.091126 2185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:17.095407 2185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:17.095722 2185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:17.096810 2185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:17.098227 2185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:17.098359 2185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.098492 2185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:17.098590 2185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.133324 2185 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:42817
I20260812 06:56:17.133384 2393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:42817 every 8 connection(s)
I20260812 06:56:17.134195 2185 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:17.138676 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 2185
I20260812 06:56:17.138895 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:17.139474 27528 external_mini_cluster.cc:1428] Setting key 667ea9f49e5e9ebf1e91f6a05b323efe
Aug 12 06:56:17 dist-test-slave-t3q3 krb5kdc[1812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517777, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:17.157338 2400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.136781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:55687 (local address 127.26.226.62:46661)
0812 06:56:17.137241 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:17.137246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:17.138185 (+ 939us) server_negotiation.cc:407] Connection header received
0812 06:56:17.140751 (+ 2566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:17.140756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:17.140825 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:17.140923 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:17.142901 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.143642 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.146197 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.146625 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.146735 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.151063 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:17.151068 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:17.151072 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:17.151104 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:17.154146 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:17.154881 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:17.154886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:17.154890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:17.155028 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:17.155468 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:17.155473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:17.155477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:17.155773 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:17.155936 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:17.156713 (+ 777us) server_negotiation.cc:299] Negotiation successful
0812 06:56:17.156939 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":334,"thread_start_us":221,"threads_started":1}
I20260812 06:56:17.158313 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.137333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:46661 (local address 127.26.226.2:55687)
0812 06:56:17.138030 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:17.138069 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:17.140435 (+ 2366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:17.141192 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:17.141204 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:17.141903 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:17.142695 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.142736 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.143826 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.143835 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.146037 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.146056 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.146865 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.146875 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.147207 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.147963 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:17.148005 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:17.150824 (+ 2819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:17.154292 (+ 3468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:17.154302 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:17.154320 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:17.154747 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:17.155145 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:17.155150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:17.155153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:17.155342 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:17.155910 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:17.155918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:17.156324 (+ 406us) client_negotiation.cc:769] Sending connection context
0812 06:56:17.156598 (+ 274us) client_negotiation.cc:240] Negotiation successful
0812 06:56:17.157134 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":284,"threads_started":1}
I20260812 06:56:17.159885 2396 heartbeater.cc:347] Connected to a master server at 127.26.226.62:46661
I20260812 06:56:17.160276 2396 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:17.161175 2396 heartbeater.cc:511] Master 127.26.226.62:46661 requested a full tablet report, sending...
I20260812 06:56:17.163053 1872 ts_manager.cc:194] Registered new tserver with Master: cbce5397c862406fabaa9363928a627c (127.26.226.2:42817)
I20260812 06:56:17.163992 1872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:55687
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:56:17.199486 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46661
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:17.548848 2404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:17.549208 2404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:17.549396 2404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:17.556903 2404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:17.557057 2404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:17.557257 2404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:17.567281 2404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:17.568748 2404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:17.570096 2404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:17.585153 2415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:17.585232 2411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:17.585595 2420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:17.586575 2404 server_base.cc:1034] running on GCE node
I20260812 06:56:17.587272 2404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:17.588363 2404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:17.589617 2404 hybrid_clock.cc:648] HybridClock initialized: now 1786517777589577 us; error 52 us; skew 500 ppm
I20260812 06:56:17.589964 2404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:17 dist-test-slave-t3q3 krb5kdc[1812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517777, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:17.593371 2404 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:56:17.594671 2404 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:17.594818 2404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:17.595171 2404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:17.598384 2404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "7583c1e74c954340af233cf77303edb5"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-t3q3"
server_key: "3fd6031fe905877ab5210effc2a44e32"
server_key_iv: "90314759675150aa89d68e77f218da54"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.599259 2404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "7583c1e74c954340af233cf77303edb5"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-t3q3"
server_key: "3fd6031fe905877ab5210effc2a44e32"
server_key_iv: "90314759675150aa89d68e77f218da54"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.604702 2404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260812 06:56:17.608521 2431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.610018 2404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:56:17.610217 2404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "7583c1e74c954340af233cf77303edb5"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-t3q3"
server_key: "3fd6031fe905877ab5210effc2a44e32"
server_key_iv: "90314759675150aa89d68e77f218da54"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.610489 2404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:17.610682 2174 heartbeater.cc:503] Master 127.26.226.62:46661 was elected leader, sending a full tablet report...
I20260812 06:56:17.658006 2404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:17.662791 2404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:17.663116 2404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:17.664279 2404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:17.665764 2404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:17.665899 2404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.666059 2404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:17.666153 2404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.704568 2404 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:37453
I20260812 06:56:17.704627 2616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:37453 every 8 connection(s)
I20260812 06:56:17.705494 2404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:17.706488 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 2404
I20260812 06:56:17.706727 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:17.707190 27528 external_mini_cluster.cc:1428] Setting key 15fc2935c32fad509f0b24d5e88e6418
Aug 12 06:56:17 dist-test-slave-t3q3 krb5kdc[1812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517777, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:17.727442 2620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.708235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:42607 (local address 127.26.226.62:46661)
0812 06:56:17.708756 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:17.708762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:17.710063 (+ 1301us) server_negotiation.cc:407] Connection header received
0812 06:56:17.711966 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:17.711970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:17.712043 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:17.712208 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:17.713749 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.714552 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.716366 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.716824 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.716929 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.720525 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:17.720531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:17.720534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:17.720573 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:17.724379 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:17.725208 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:17.725213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:17.725215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:17.725349 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:17.725772 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:17.725775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:17.725777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:17.726073 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:17.726191 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:17.726954 (+ 763us) server_negotiation.cc:299] Negotiation successful
0812 06:56:17.727154 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":210,"threads_started":1}
I20260812 06:56:17.728559 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.708932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:46661 (local address 127.26.226.3:42607)
0812 06:56:17.709904 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:17.709961 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:17.711711 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:17.712410 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:17.712418 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:17.712981 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:17.713553 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.713570 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.714701 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.714706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.716208 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.716222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.717023 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.717027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.717235 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.717965 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:17.717994 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:17.720293 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:17.724529 (+ 4236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:17.724537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:17.724552 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:17.725059 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:17.725464 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:17.725467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:17.725469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:17.725662 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:17.726202 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:17.726208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:17.726577 (+ 369us) client_negotiation.cc:769] Sending connection context
0812 06:56:17.726825 (+ 248us) client_negotiation.cc:240] Negotiation successful
0812 06:56:17.727251 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":428,"threads_started":1}
I20260812 06:56:17.730090 2617 heartbeater.cc:347] Connected to a master server at 127.26.226.62:46661
I20260812 06:56:17.730389 2617 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:17.731027 2617 heartbeater.cc:511] Master 127.26.226.62:46661 requested a full tablet report, sending...
I20260812 06:56:17.732481 1872 ts_manager.cc:194] Registered new tserver with Master: 7583c1e74c954340af233cf77303edb5 (127.26.226.3:37453)
I20260812 06:56:17.733371 1872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:42607
I20260812 06:56:17.736367 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:56:17.753873 2620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.741146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59944 (local address 127.26.226.62:46661)
0812 06:56:17.741350 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:17.741358 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:17.741776 (+ 418us) server_negotiation.cc:407] Connection header received
0812 06:56:17.741980 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:17.741987 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:17.742060 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:17.742166 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:17.743657 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.744475 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.745775 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.746206 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.746349 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.748670 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:17.748675 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:17.748678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:17.748710 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:17.751228 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:17.751878 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:17.751881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:17.751884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:17.752015 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:17.752444 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:17.752449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:17.752451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:17.752823 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:17.753036 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:17.753305 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 06:56:17.753498 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:56:17.758172 1872 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:17.765272 1872 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:17.798476 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.778180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:37453 (local address 127.0.0.1:56904)
0812 06:56:17.788886 (+ 10706us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:17.788907 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:17.789036 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:17.789895 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:17.789904 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:17.789939 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:17.790488 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.790507 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.792907 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.792913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.795439 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.795454 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.797443 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.797449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.797643 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.797712 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 06:56:17.797838 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 06:56:17.798054 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10586,"thread_start_us":10421,"threads_started":1}
I20260812 06:56:17.798915 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.777887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:42817 (local address 127.0.0.1:46522)
0812 06:56:17.787948 (+ 10061us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:17.788001 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:17.788241 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:17.791371 (+ 3130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:17.791378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:17.791405 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:17.791842 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.791855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.793991 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.793997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.796227 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.796242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.798120 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.798125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.798326 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.798356 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:56:17.798432 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 06:56:17.798576 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9798,"thread_start_us":9715,"threads_started":1}
I20260812 06:56:17.799925 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.778166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.26.226.3:37453)
0812 06:56:17.787570 (+ 9404us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:17.787581 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:17.789167 (+ 1586us) server_negotiation.cc:407] Connection header received
0812 06:56:17.789284 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:17.789294 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:17.789581 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:17.790732 (+ 1151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:17.791078 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.792743 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.795658 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.797206 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.799042 (+ 1836us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.799300 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:17.799382 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 06:56:17.799549 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9187,"thread_start_us":8973,"threads_started":1}
I20260812 06:56:17.800221 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.778343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46522 (local address 127.26.226.2:42817)
0812 06:56:17.789685 (+ 11342us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:17.789696 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:17.790769 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 06:56:17.790886 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:17.790893 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:17.791117 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:17.792051 (+ 934us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:17.792326 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.793826 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.796448 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.797880 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.799422 (+ 1542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.799596 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:17.799664 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 06:56:17.799841 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11168,"thread_start_us":11047,"threads_started":1}
I20260812 06:56:17.804340 2287 tablet_service.cc:1467] Processing CreateTablet for tablet 6d007ce76ae14c0aa0e600977edb2a60 (DEFAULT_TABLE table=test-table [id=5ba428bf03314e669dcd08f73935717e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:17.804351 2510 tablet_service.cc:1467] Processing CreateTablet for tablet 6d007ce76ae14c0aa0e600977edb2a60 (DEFAULT_TABLE table=test-table [id=5ba428bf03314e669dcd08f73935717e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:17.805797 2510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d007ce76ae14c0aa0e600977edb2a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:17.805827 2287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d007ce76ae14c0aa0e600977edb2a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:17.808576 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.789751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:45087 (local address 127.0.0.1:46836)
0812 06:56:17.798743 (+ 8992us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:17.798762 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:17.798855 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:17.801644 (+ 2789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:17.801648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:17.801674 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:17.801826 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.801836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.803717 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.803723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.805769 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.805803 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.807675 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.807681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.807866 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.807901 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 06:56:17.808005 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 06:56:17.808269 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8883,"thread_start_us":11276,"threads_started":1}
I20260812 06:56:17.808638 2645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.789634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.26.226.1:45087)
0812 06:56:17.800987 (+ 11353us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:17.800997 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:17.801029 (+ 32us) server_negotiation.cc:407] Connection header received
0812 06:56:17.801146 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:17.801154 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:17.801420 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:17.801643 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:17.801989 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.803543 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.805993 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.807455 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.807650 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.807881 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:17.808120 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 06:56:17.808289 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11167,"thread_start_us":10996,"threads_started":1}
I20260812 06:56:17.812831 2068 tablet_service.cc:1467] Processing CreateTablet for tablet 6d007ce76ae14c0aa0e600977edb2a60 (DEFAULT_TABLE table=test-table [id=5ba428bf03314e669dcd08f73935717e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:17.814419 2068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d007ce76ae14c0aa0e600977edb2a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:17.819721 2647 tablet_bootstrap.cc:492] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: Bootstrap starting.
I20260812 06:56:17.821136 2648 tablet_bootstrap.cc:492] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: Bootstrap starting.
I20260812 06:56:17.823935 2647 tablet_bootstrap.cc:654] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:17.824896 2648 tablet_bootstrap.cc:654] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:17.825423 2647 log.cc:824] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:17.826231 2648 log.cc:824] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:17.828967 2647 tablet_bootstrap.cc:492] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: No bootstrap required, opened a new log
I20260812 06:56:17.829346 2647 ts_tablet_manager.cc:1397] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260812 06:56:17.829380 2648 tablet_bootstrap.cc:492] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: No bootstrap required, opened a new log
I20260812 06:56:17.829653 2648 ts_tablet_manager.cc:1397] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 06:56:17.830425 2652 tablet_bootstrap.cc:492] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: Bootstrap starting.
I20260812 06:56:17.834193 2652 tablet_bootstrap.cc:654] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:17.835541 2652 log.cc:824] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:17.837671 2647 raft_consensus.cc:348] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } }
I20260812 06:56:17.838126 2647 raft_consensus.cc:374] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:17.838258 2647 raft_consensus.cc:729] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbce5397c862406fabaa9363928a627c, State: Initialized, Role: FOLLOWER
I20260812 06:56:17.839103 2652 tablet_bootstrap.cc:492] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: No bootstrap required, opened a new log
I20260812 06:56:17.838953 2647 consensus_queue.cc:260] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } }
I20260812 06:56:17.839475 2652 ts_tablet_manager.cc:1397] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260812 06:56:17.840377 2648 raft_consensus.cc:348] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } }
I20260812 06:56:17.840842 2648 raft_consensus.cc:374] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:17.840976 2648 raft_consensus.cc:729] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7583c1e74c954340af233cf77303edb5, State: Initialized, Role: FOLLOWER
I20260812 06:56:17.841579 2648 consensus_queue.cc:260] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } }
I20260812 06:56:17.842886 2617 heartbeater.cc:503] Master 127.26.226.62:46661 was elected leader, sending a full tablet report...
I20260812 06:56:17.843472 2648 ts_tablet_manager.cc:1428] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260812 06:56:17.844846 2396 heartbeater.cc:503] Master 127.26.226.62:46661 was elected leader, sending a full tablet report...
I20260812 06:56:17.845472 2647 ts_tablet_manager.cc:1428] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260812 06:56:17.851132 2652 raft_consensus.cc:348] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } }
I20260812 06:56:17.851616 2652 raft_consensus.cc:374] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:17.851712 2652 raft_consensus.cc:729] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3194b2e374479f9192145ad0d4b78a, State: Initialized, Role: FOLLOWER
I20260812 06:56:17.852298 2652 consensus_queue.cc:260] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } }
I20260812 06:56:17.853969 2652 ts_tablet_manager.cc:1428] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: Time spent starting tablet: real 0.014s user 0.010s sys 0.002s
W20260812 06:56:17.886161 2397 tablet.cc:2369] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:56:17.957195 2619 tablet.cc:2369] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:18.058070 2655 raft_consensus.cc:482] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:18.058491 2655 raft_consensus.cc:504] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } }
I20260812 06:56:18.060333 2655 leader_election.cc:288] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7583c1e74c954340af233cf77303edb5 (127.26.226.3:37453), 5a3194b2e374479f9192145ad0d4b78a (127.26.226.1:45087)
I20260812 06:56:18.068023 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.060846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:53715 (local address 127.26.226.3:37453)
0812 06:56:18.061063 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.061068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.061591 (+ 523us) server_negotiation.cc:407] Connection header received
0812 06:56:18.061800 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.061806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.061896 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.062030 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:18.062887 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.063924 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.066089 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.066998 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.067135 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.067192 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.067620 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.067762 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:56:18.068133 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.060697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:37453 (local address 127.26.226.2:53715)
0812 06:56:18.061432 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:18.061461 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:18.061630 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:18.062222 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:18.062226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:18.062256 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:18.062710 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.062727 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.064113 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.064119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.065919 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.065932 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.067241 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.067248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.067427 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.067465 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:56:18.067574 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 06:56:18.067723 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":25,"thread_start_us":371,"threads_started":1}
I20260812 06:56:18.068636 2645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.061783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:44993 (local address 127.26.226.1:45087)
0812 06:56:18.062049 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.062055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.062376 (+ 321us) server_negotiation.cc:407] Connection header received
0812 06:56:18.062574 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.062581 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.062669 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.062782 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:18.063543 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.064485 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.066483 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.067429 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.067604 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.067674 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.068190 (+ 516us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.068346 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:56:18.069339 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.061548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:45087 (local address 127.26.226.2:44993)
0812 06:56:18.062248 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:18.062263 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:18.062419 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:18.062865 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:18.062869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:18.062898 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:18.063340 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.063353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.064625 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.064629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.066225 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.066240 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.067696 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.067712 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.067910 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.067947 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 06:56:18.068830 (+ 883us) client_negotiation.cc:240] Negotiation successful
0812 06:56:18.068998 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":38,"thread_start_us":379,"threads_started":1}
I20260812 06:56:18.069331 2547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d007ce76ae14c0aa0e600977edb2a60" candidate_uuid: "cbce5397c862406fabaa9363928a627c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7583c1e74c954340af233cf77303edb5" is_pre_election: true
I20260812 06:56:18.070065 2547 raft_consensus.cc:2454] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbce5397c862406fabaa9363928a627c in term 0.
I20260812 06:56:18.070509 2099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d007ce76ae14c0aa0e600977edb2a60" candidate_uuid: "cbce5397c862406fabaa9363928a627c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3194b2e374479f9192145ad0d4b78a" is_pre_election: true
I20260812 06:56:18.070766 2657 raft_consensus.cc:482] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:18.070791 2209 leader_election.cc:302] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7583c1e74c954340af233cf77303edb5, cbce5397c862406fabaa9363928a627c; no voters:
I20260812 06:56:18.071240 2655 raft_consensus.cc:2781] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:18.071058 2657 raft_consensus.cc:504] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } }
I20260812 06:56:18.071396 2655 raft_consensus.cc:482] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:18.071552 2655 raft_consensus.cc:3037] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:18.071908 2099 raft_consensus.cc:2454] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbce5397c862406fabaa9363928a627c in term 0.
I20260812 06:56:18.073158 2657 leader_election.cc:288] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7583c1e74c954340af233cf77303edb5 (127.26.226.3:37453), cbce5397c862406fabaa9363928a627c (127.26.226.2:42817)
I20260812 06:56:18.074672 2655 raft_consensus.cc:504] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } }
I20260812 06:56:18.075512 2655 leader_election.cc:288] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [CANDIDATE]: Term 1 election: Requested vote from peers 7583c1e74c954340af233cf77303edb5 (127.26.226.3:37453), 5a3194b2e374479f9192145ad0d4b78a (127.26.226.1:45087)
I20260812 06:56:18.076082 2547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d007ce76ae14c0aa0e600977edb2a60" candidate_uuid: "cbce5397c862406fabaa9363928a627c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7583c1e74c954340af233cf77303edb5"
I20260812 06:56:18.076334 2547 raft_consensus.cc:3037] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:18.076311 2099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d007ce76ae14c0aa0e600977edb2a60" candidate_uuid: "cbce5397c862406fabaa9363928a627c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3194b2e374479f9192145ad0d4b78a"
I20260812 06:56:18.076514 2099 raft_consensus.cc:3037] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:18.079988 2547 raft_consensus.cc:2454] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbce5397c862406fabaa9363928a627c in term 1.
I20260812 06:56:18.080240 2099 raft_consensus.cc:2454] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbce5397c862406fabaa9363928a627c in term 1.
I20260812 06:56:18.080452 2209 leader_election.cc:302] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7583c1e74c954340af233cf77303edb5, cbce5397c862406fabaa9363928a627c; no voters:
W20260812 06:56:18.080976 2178 tablet.cc:2369] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:18.081725 2655 raft_consensus.cc:2781] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:18.082248 2655 raft_consensus.cc:686] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 1 LEADER]: Becoming Leader. State: Replica: cbce5397c862406fabaa9363928a627c, State: Running, Role: LEADER
I20260812 06:56:18.082633 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.073737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:37325 (local address 127.26.226.3:37453)
0812 06:56:18.073899 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.073905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.074160 (+ 255us) server_negotiation.cc:407] Connection header received
0812 06:56:18.074509 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.074517 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.074600 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.074665 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:18.075510 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.076820 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.080050 (+ 3230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.081365 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.081497 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.081577 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.082184 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.082344 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:56:18.082816 2655 consensus_queue.cc:237] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } }
I20260812 06:56:18.083418 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.073308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:37453 (local address 127.26.226.1:37325)
0812 06:56:18.074055 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:18.074076 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:18.074303 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:18.074887 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:18.074893 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:18.074924 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:18.075342 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.075359 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.077002 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.077007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.079839 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.079860 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.081797 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.081803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.081981 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.082017 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:56:18.082870 (+ 853us) client_negotiation.cc:240] Negotiation successful
0812 06:56:18.083012 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":73,"thread_start_us":805,"threads_started":1}
I20260812 06:56:18.083580 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.074031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:34977 (local address 127.26.226.2:42817)
0812 06:56:18.074217 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.074223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.074853 (+ 630us) server_negotiation.cc:407] Connection header received
0812 06:56:18.075218 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.075226 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.075313 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.075415 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:18.076402 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.077417 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.080748 (+ 3331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.082049 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.082176 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.082253 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.083172 (+ 919us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.083305 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 06:56:18.084340 2547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d007ce76ae14c0aa0e600977edb2a60" candidate_uuid: "5a3194b2e374479f9192145ad0d4b78a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7583c1e74c954340af233cf77303edb5" is_pre_election: true
I20260812 06:56:18.084471 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.073432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:42817 (local address 127.26.226.1:34977)
0812 06:56:18.074755 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:18.074775 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:18.075047 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:18.075580 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:18.075648 (+ 68us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:18.075677 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:18.076154 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.076170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.077687 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.077692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.080482 (+ 2790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.080499 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.082553 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.082558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.082932 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.082963 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:56:18.083907 (+ 944us) client_negotiation.cc:240] Negotiation successful
0812 06:56:18.084042 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1199,"mutex_wait_us":18,"spinlock_wait_cycles":1280,"thread_start_us":988,"threads_started":1}
I20260812 06:56:18.084794 2547 raft_consensus.cc:2379] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a3194b2e374479f9192145ad0d4b78a in current term 1: Already voted for candidate cbce5397c862406fabaa9363928a627c in this term.
I20260812 06:56:18.085409 2322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d007ce76ae14c0aa0e600977edb2a60" candidate_uuid: "5a3194b2e374479f9192145ad0d4b78a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbce5397c862406fabaa9363928a627c" is_pre_election: true
I20260812 06:56:18.086437 1995 leader_election.cc:302] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a3194b2e374479f9192145ad0d4b78a; no voters: 7583c1e74c954340af233cf77303edb5, cbce5397c862406fabaa9363928a627c
I20260812 06:56:18.086843 2657 raft_consensus.cc:2726] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 06:56:18.089512 1872 catalog_manager.cc:5495] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c reported cstate change: term changed from 0 to 1, leader changed from <none> to cbce5397c862406fabaa9363928a627c (127.26.226.2). New cstate: current_term: 1 leader_uuid: "cbce5397c862406fabaa9363928a627c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbce5397c862406fabaa9363928a627c" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 } health_report { overall_health: UNKNOWN } } }
I20260812 06:56:18.169334 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.163646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46528 (local address 127.26.226.2:42817)
0812 06:56:18.163889 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.163898 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.164007 (+ 109us) server_negotiation.cc:407] Connection header received
0812 06:56:18.164272 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.164276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.164377 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.164519 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:18.164906 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.165706 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.167070 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.167490 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.167599 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.168125 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:18.168666 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:18.168777 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.168896 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.168965 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 06:56:18.178011 2547 raft_consensus.cc:1261] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [term 1 FOLLOWER]: Refusing update from remote peer cbce5397c862406fabaa9363928a627c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:18.178102 2099 raft_consensus.cc:1261] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 1 FOLLOWER]: Refusing update from remote peer cbce5397c862406fabaa9363928a627c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:18.179040 2668 consensus_queue.cc:1035] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7583c1e74c954340af233cf77303edb5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:18.179513 2655 consensus_queue.cc:1035] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3194b2e374479f9192145ad0d4b78a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 45087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:18.192844 2676 mvcc.cc:205] Tried to move back new op lower bound from 7317576819405852672 to 7317576819032420352. Current Snapshot: MvccSnapshot[applied={T|T < 7317576819405852672}]
I20260812 06:56:18.193800 2683 mvcc.cc:205] Tried to move back new op lower bound from 7317576819405852672 to 7317576819032420352. Current Snapshot: MvccSnapshot[applied={T|T < 7317576819405852672}]
I20260812 06:56:18.209015 1871 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:18.209355 1871 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:18.213603 1871 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4fb07cbacbb44feeaea328ea2efb9c8f: Sending DeleteTablet for 3 replicas of tablet 6d007ce76ae14c0aa0e600977edb2a60
I20260812 06:56:18.214987 2287 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d007ce76ae14c0aa0e600977edb2a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:18 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:46522
I20260812 06:56:18.214990 2510 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d007ce76ae14c0aa0e600977edb2a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:18 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:56904
I20260812 06:56:18.215445 2068 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d007ce76ae14c0aa0e600977edb2a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:18 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:46836
I20260812 06:56:18.218238 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 1968
I20260812 06:56:18.226135 2699 tablet_replica.cc:330] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: stopping tablet replica
I20260812 06:56:18.226974 2699 raft_consensus.cc:2229] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:18.227679 2695 tablet_replica.cc:330] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: stopping tablet replica
I20260812 06:56:18.227751 2696 tablet_replica.cc:330] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: stopping tablet replica
I20260812 06:56:18.228123 2695 raft_consensus.cc:2229] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:18.228219 2696 raft_consensus.cc:2229] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:18.228919 2696 raft_consensus.cc:2258] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:18.229027 2695 raft_consensus.cc:2258] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:18.229714 2699 raft_consensus.cc:2258] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:18.231488 2695 ts_tablet_manager.cc:1905] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:18.232359 2699 ts_tablet_manager.cc:1905] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:18.239435 2695 ts_tablet_manager.cc:1918] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:18.239614 2695 log.cc:1196] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/6d007ce76ae14c0aa0e600977edb2a60
I20260812 06:56:18.240310 2695 ts_tablet_manager.cc:1939] T 6d007ce76ae14c0aa0e600977edb2a60 P cbce5397c862406fabaa9363928a627c: Deleting consensus metadata
I20260812 06:56:18.241364 2696 ts_tablet_manager.cc:1905] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:18.242249 1847 catalog_manager.cc:4841] TS cbce5397c862406fabaa9363928a627c (127.26.226.2:42817): tablet 6d007ce76ae14c0aa0e600977edb2a60 (table test-table [id=5ba428bf03314e669dcd08f73935717e]) successfully deleted
I20260812 06:56:18.239990 2699 ts_tablet_manager.cc:1918] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:18.244333 2699 log.cc:1196] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/6d007ce76ae14c0aa0e600977edb2a60
I20260812 06:56:18.245029 2699 ts_tablet_manager.cc:1939] T 6d007ce76ae14c0aa0e600977edb2a60 P 5a3194b2e374479f9192145ad0d4b78a: Deleting consensus metadata
Aug 12 06:56:18 dist-test-slave-t3q3 krb5kdc[1812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:56:18.247152 1849 catalog_manager.cc:4841] TS 5a3194b2e374479f9192145ad0d4b78a (127.26.226.1:45087): tablet 6d007ce76ae14c0aa0e600977edb2a60 (table test-table [id=5ba428bf03314e669dcd08f73935717e]) successfully deleted
I20260812 06:56:18.249718 2696 ts_tablet_manager.cc:1918] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:18.249941 2696 log.cc:1196] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/6d007ce76ae14c0aa0e600977edb2a60
I20260812 06:56:18.250587 2696 ts_tablet_manager.cc:1939] T 6d007ce76ae14c0aa0e600977edb2a60 P 7583c1e74c954340af233cf77303edb5: Deleting consensus metadata
I20260812 06:56:18.252842 1849 catalog_manager.cc:4841] TS 7583c1e74c954340af233cf77303edb5 (127.26.226.3:37453): tablet 6d007ce76ae14c0aa0e600977edb2a60 (table test-table [id=5ba428bf03314e669dcd08f73935717e]) successfully deleted
I20260812 06:56:18.255329 2645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.220084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.26.226.1:45087)
0812 06:56:18.220329 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.220340 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.230690 (+ 10350us) server_negotiation.cc:407] Connection header received
0812 06:56:18.230946 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.230952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.231047 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.231189 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:18.233310 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.234611 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.236826 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.237619 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.237746 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.247129 (+ 9383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:18.247145 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:18.247166 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:18.247233 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:18.251759 (+ 4526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:18.252908 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:18.252920 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:18.252923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:18.253168 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:18.253622 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:18.253626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:18.253628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:18.254074 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:18.254495 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.254782 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.254978 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 06:56:18.256050 2164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:18.355317 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 1968
I20260812 06:56:18.387738 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 2185
Aug 12 06:56:18 dist-test-slave-t3q3 krb5kdc[1812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:56:18.405972 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.388664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46536 (local address 127.26.226.2:42817)
0812 06:56:18.388854 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.388864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.388886 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:56:18.389108 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.389113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.389203 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.389330 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:18.390626 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.391374 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.393114 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.393604 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.393779 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.398483 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:18.398504 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:18.398526 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:18.398578 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:18.402821 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:18.403636 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:18.403646 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:18.403649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:18.403840 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:18.404325 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:18.404331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:18.404334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:18.404804 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:18.405156 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.405330 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.405591 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:56:18.407014 2387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:18.524933 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 2185
I20260812 06:56:18.551334 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 2404
Aug 12 06:56:18 dist-test-slave-t3q3 krb5kdc[1812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:56:18.566972 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.552208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56920 (local address 127.26.226.3:37453)
0812 06:56:18.552384 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.552388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.552407 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:56:18.552529 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.552533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.552598 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.552687 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:18.554028 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.554750 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.555988 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.556637 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.556742 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.561174 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:18.561193 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:18.561210 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:18.561247 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:18.564363 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:18.565059 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:18.565067 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:18.565069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:18.565200 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:18.565536 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:18.565539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:18.565541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:18.565917 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:18.566201 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.566432 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.566594 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:56:18.567711 2609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:18.657805 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 2404
I20260812 06:56:18.684921 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 1828
I20260812 06:56:18.686172 1920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:18.802474 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 1828
2026-08-12T06:56:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4938 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T06:56:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:18Z Disabled control of system clock
I20260812 06:56:18.890971 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:44545
--webserver_interface=127.26.226.62
--webserver_port=0
--builtin_ntp_servers=127.26.226.20:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:44545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 06:56:19.237349 2712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:19.237679 2712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:19.237771 2712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:19.245242 2712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:19.245366 2712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:19.245406 2712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:19.245458 2712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:19.245504 2712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:19.255214 2712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:44545
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:44545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:19.256816 2712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:19.258688 2712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:19.271538 2722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:19.271430 2723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:19.271479 2726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:19.272378 2712 server_base.cc:1034] running on GCE node
I20260812 06:56:19.273290 2712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:19.274959 2712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:19.276216 2712 hybrid_clock.cc:648] HybridClock initialized: now 1786517779276144 us; error 60 us; skew 500 ppm
I20260812 06:56:19.276630 2712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:19.279320 2712 webserver.cc:460] Webserver started at http://127.26.226.62:40201/ using document root <none> and password file <none>
I20260812 06:56:19.280175 2712 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:19.280318 2712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:19.280678 2712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:19.283777 2712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "b96470b0e21946af9f056668ba56b43f"
format_stamp: "Formatted at 2026-08-12 06:56:19 on dist-test-slave-t3q3"
server_key: "1bc874af5b37835517e2bdd1e564f577"
server_key_iv: "df83960b9e74f54578a2e0c8ea8fb644"
server_key_version: "encryptionkey@0"
I20260812 06:56:19.284714 2712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "b96470b0e21946af9f056668ba56b43f"
format_stamp: "Formatted at 2026-08-12 06:56:19 on dist-test-slave-t3q3"
server_key: "1bc874af5b37835517e2bdd1e564f577"
server_key_iv: "df83960b9e74f54578a2e0c8ea8fb644"
server_key_version: "encryptionkey@0"
I20260812 06:56:19.290503 2712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 06:56:19.294464 2733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:19.296274 2712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 06:56:19.296509 2712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "b96470b0e21946af9f056668ba56b43f"
format_stamp: "Formatted at 2026-08-12 06:56:19 on dist-test-slave-t3q3"
server_key: "1bc874af5b37835517e2bdd1e564f577"
server_key_iv: "df83960b9e74f54578a2e0c8ea8fb644"
server_key_version: "encryptionkey@0"
I20260812 06:56:19.296806 2712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:19.303654 2712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:19.304692 2712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:19.305070 2712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:19.326632 2712 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:44545
I20260812 06:56:19.326668 2809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:44545 every 8 connection(s)
I20260812 06:56:19.328263 2712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:19.332293 2810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:19.336529 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 2712
I20260812 06:56:19.336755 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:19.337423 27528 external_mini_cluster.cc:1428] Setting key 31e25e85711da97f3dc897fbcf4edf5d
I20260812 06:56:19.345424 2810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f: Bootstrap starting.
I20260812 06:56:19.347707 2813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:19.339610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41712 (local address 127.26.226.62:44545)
0812 06:56:19.340803 (+ 1193us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:19.340823 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:19.340881 (+ 58us) server_negotiation.cc:407] Connection header received
0812 06:56:19.343431 (+ 2550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:19.343462 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:19.344034 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:19.344526 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:19.345592 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:19.345601 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:19.345617 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:19.345634 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:19.345730 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:19.345757 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:19.345987 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:19.346143 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 06:56:19.346404 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":389,"threads_started":1}
I20260812 06:56:19.349587 2810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:19.350823 2810 log.cc:824] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:19.353958 2810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f: No bootstrap required, opened a new log
I20260812 06:56:19.361979 2810 raft_consensus.cc:348] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96470b0e21946af9f056668ba56b43f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44545 } }
I20260812 06:56:19.362411 2810 raft_consensus.cc:374] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:19.362531 2810 raft_consensus.cc:729] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96470b0e21946af9f056668ba56b43f, State: Initialized, Role: FOLLOWER
I20260812 06:56:19.363210 2810 consensus_queue.cc:260] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96470b0e21946af9f056668ba56b43f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44545 } }
I20260812 06:56:19.363458 2810 raft_consensus.cc:388] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:19.363612 2810 raft_consensus.cc:482] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:19.363811 2810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:19.366472 2810 raft_consensus.cc:504] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96470b0e21946af9f056668ba56b43f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44545 } }
I20260812 06:56:19.367050 2810 leader_election.cc:302] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b96470b0e21946af9f056668ba56b43f; no voters:
I20260812 06:56:19.367659 2810 leader_election.cc:288] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:19.367815 2816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:19.368635 2816 raft_consensus.cc:686] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [term 1 LEADER]: Becoming Leader. State: Replica: b96470b0e21946af9f056668ba56b43f, State: Running, Role: LEADER
I20260812 06:56:19.369182 2816 consensus_queue.cc:237] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96470b0e21946af9f056668ba56b43f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44545 } }
I20260812 06:56:19.370018 2810 sys_catalog.cc:564] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:19.373005 2817 sys_catalog.cc:455] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b96470b0e21946af9f056668ba56b43f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96470b0e21946af9f056668ba56b43f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44545 } } }
I20260812 06:56:19.372920 2818 sys_catalog.cc:455] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b96470b0e21946af9f056668ba56b43f. Latest consensus state: current_term: 1 leader_uuid: "b96470b0e21946af9f056668ba56b43f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96470b0e21946af9f056668ba56b43f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44545 } } }
I20260812 06:56:19.373445 2818 sys_catalog.cc:458] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:19.373448 2817 sys_catalog.cc:458] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:19.375866 2832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:19.381696 2832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:19.391032 2832 catalog_manager.cc:1347] Generated new cluster ID: af4133d235724a2da40ad2feda6f829f
I20260812 06:56:19.391201 2832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:19.414230 2832 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:19.414676 2832 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:19.427451 2832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f: Generated new TSK 0
I20260812 06:56:19.428460 2832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:19.448565 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:44545
--builtin_ntp_servers=127.26.226.20:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 06:56:19.803411 2846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:19.803771 2846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:19.803937 2846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:19.811591 2846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:19.811717 2846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:19.811901 2846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:19.821693 2846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:44545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:19.823231 2846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:19.825241 2846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:19.839350 2855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:19.839320 2856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:19.839497 2863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:19.840541 2846 server_base.cc:1034] running on GCE node
I20260812 06:56:19.841208 2846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:19.842170 2846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:19.843403 2846 hybrid_clock.cc:648] HybridClock initialized: now 1786517779843356 us; error 61 us; skew 500 ppm
I20260812 06:56:19.843751 2846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:19.846189 2846 webserver.cc:460] Webserver started at http://127.26.226.1:42343/ using document root <none> and password file <none>
I20260812 06:56:19.846892 2846 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:19.847035 2846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:19.847379 2846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:19.850476 2846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "8edbfb1552024ebb91b6c6f2bdba5918"
format_stamp: "Formatted at 2026-08-12 06:56:19 on dist-test-slave-t3q3"
server_key: "3263cb65d8e921035ec53bb39b646d2f"
server_key_iv: "c14bd25a8b562a1ff0d8d66abb5347a4"
server_key_version: "encryptionkey@0"
I20260812 06:56:19.851286 2846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "8edbfb1552024ebb91b6c6f2bdba5918"
format_stamp: "Formatted at 2026-08-12 06:56:19 on dist-test-slave-t3q3"
server_key: "3263cb65d8e921035ec53bb39b646d2f"
server_key_iv: "c14bd25a8b562a1ff0d8d66abb5347a4"
server_key_version: "encryptionkey@0"
I20260812 06:56:19.856568 2846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 06:56:19.859975 2871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:19.861683 2846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:19.861860 2846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "8edbfb1552024ebb91b6c6f2bdba5918"
format_stamp: "Formatted at 2026-08-12 06:56:19 on dist-test-slave-t3q3"
server_key: "3263cb65d8e921035ec53bb39b646d2f"
server_key_iv: "c14bd25a8b562a1ff0d8d66abb5347a4"
server_key_version: "encryptionkey@0"
I20260812 06:56:19.862095 2846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:19.867942 2846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:19.868793 2846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:19.869087 2846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:19.870039 2846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:19.871528 2846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:19.871656 2846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:19.871784 2846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:19.871872 2846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:19.909247 2846 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:43567
I20260812 06:56:19.909307 3082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:43567 every 8 connection(s)
I20260812 06:56:19.910609 2846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:19.915320 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 2846
I20260812 06:56:19.915527 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:19.916224 27528 external_mini_cluster.cc:1428] Setting key 1849e14ff2c30b2974ef1199b14e4705
I20260812 06:56:19.922423 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:44545
--builtin_ntp_servers=127.26.226.20:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 06:56:19.923002 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:19.913543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:33451 (local address 127.26.226.62:44545)
0812 06:56:19.920476 (+ 6933us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:19.920482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:19.920507 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:56:19.920559 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:19.920564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:19.920637 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:19.920767 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:19.922167 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:19.922173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:19.922177 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:19.922196 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:19.922248 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:19.922261 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:19.922395 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:19.922607 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 06:56:19.922675 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6810,"thread_start_us":6704,"threads_started":1}
I20260812 06:56:19.923897 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:19.914129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:44545 (local address 127.26.226.1:33451)
0812 06:56:19.915454 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:19.915520 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:19.918548 (+ 3028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:19.921024 (+ 2476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:19.921033 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:19.921125 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:19.921855 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:19.921857 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:19.921896 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:19.921983 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:19.922363 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:19.922377 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:19.922385 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 06:56:19.922682 (+ 297us) client_negotiation.cc:240] Negotiation successful
0812 06:56:19.922967 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":339,"threads_started":1}
I20260812 06:56:19.925525 3083 heartbeater.cc:347] Connected to a master server at 127.26.226.62:44545
I20260812 06:56:19.925854 3083 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:19.926404 3083 heartbeater.cc:511] Master 127.26.226.62:44545 requested a full tablet report, sending...
I20260812 06:56:19.928732 2757 ts_manager.cc:194] Registered new tserver with Master: 8edbfb1552024ebb91b6c6f2bdba5918 (127.26.226.1:43567)
W20260812 06:56:20.260164 3094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:20.260450 3094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:20.260635 3094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:20.268110 3094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:20.268211 3094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:20.268411 3094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:20.277912 3094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:44545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:20.279234 3094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:20.281044 3094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:20.294229 3099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:20.294269 3100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:20.294948 3104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:20.295254 3094 server_base.cc:1034] running on GCE node
I20260812 06:56:20.295950 3094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:20.298889 3094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:20.300181 3094 hybrid_clock.cc:648] HybridClock initialized: now 1786517780300118 us; error 43 us; skew 500 ppm
I20260812 06:56:20.300539 3094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:20.302899 3094 webserver.cc:460] Webserver started at http://127.26.226.2:46527/ using document root <none> and password file <none>
I20260812 06:56:20.303634 3094 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:20.303777 3094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:20.304245 3094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:20.307235 3094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "93d4730d1f5049ddad9ebec19da393f1"
format_stamp: "Formatted at 2026-08-12 06:56:20 on dist-test-slave-t3q3"
server_key: "8a4111d16d99c3f4b9dfe3ef145be712"
server_key_iv: "cab406f1d18ebe5fc60c913fe7da83d1"
server_key_version: "encryptionkey@0"
I20260812 06:56:20.307988 3094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "93d4730d1f5049ddad9ebec19da393f1"
format_stamp: "Formatted at 2026-08-12 06:56:20 on dist-test-slave-t3q3"
server_key: "8a4111d16d99c3f4b9dfe3ef145be712"
server_key_iv: "cab406f1d18ebe5fc60c913fe7da83d1"
server_key_version: "encryptionkey@0"
I20260812 06:56:20.313364 3094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 06:56:20.316721 3114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:20.318009 3094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:20.318218 3094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "93d4730d1f5049ddad9ebec19da393f1"
format_stamp: "Formatted at 2026-08-12 06:56:20 on dist-test-slave-t3q3"
server_key: "8a4111d16d99c3f4b9dfe3ef145be712"
server_key_iv: "cab406f1d18ebe5fc60c913fe7da83d1"
server_key_version: "encryptionkey@0"
I20260812 06:56:20.318444 3094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:20.324290 3094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:20.325083 3094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:20.325362 3094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:20.326534 3094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:20.328307 3094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:20.328429 3094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:20.328532 3094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:20.328604 3094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:20.363366 3094 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:36483
I20260812 06:56:20.363425 3285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:36483 every 8 connection(s)
I20260812 06:56:20.364774 3094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:20.367344 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 3094
I20260812 06:56:20.367537 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:20.368230 27528 external_mini_cluster.cc:1428] Setting key a06b3bfb47b3e9de93f5c9c53e71cd38
I20260812 06:56:20.371078 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:44545
--builtin_ntp_servers=127.26.226.20:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 06:56:20.374435 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.368014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:55003 (local address 127.26.226.62:44545)
0812 06:56:20.368274 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:20.368282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:20.369922 (+ 1640us) server_negotiation.cc:407] Connection header received
0812 06:56:20.371902 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:20.371908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:20.371983 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:20.372138 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:20.373591 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:20.373598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:20.373605 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:20.373630 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:20.373671 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:20.373684 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:20.373832 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:20.374058 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 06:56:20.374135 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260812 06:56:20.375502 3290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.368603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:44545 (local address 127.26.226.2:55003)
0812 06:56:20.369767 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:20.369812 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:20.371636 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:20.372364 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:20.372373 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:20.372463 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:20.373230 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:20.373233 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:20.373275 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:20.373368 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:20.373803 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:20.373816 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:20.373825 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:56:20.374150 (+ 325us) client_negotiation.cc:240] Negotiation successful
0812 06:56:20.374415 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":453,"threads_started":1}
I20260812 06:56:20.377072 3287 heartbeater.cc:347] Connected to a master server at 127.26.226.62:44545
I20260812 06:56:20.377398 3287 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:20.378108 3287 heartbeater.cc:511] Master 127.26.226.62:44545 requested a full tablet report, sending...
I20260812 06:56:20.379751 2757 ts_manager.cc:194] Registered new tserver with Master: 93d4730d1f5049ddad9ebec19da393f1 (127.26.226.2:36483)
W20260812 06:56:20.721699 3292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:20.721972 3292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:20.722067 3292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:20.730151 3292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:20.730288 3292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:20.730513 3292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:20.742632 3292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:44545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:20.743866 3292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:20.745659 3292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:20.759064 3304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:20.759097 3300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:20.759145 3298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:20.761392 3292 server_base.cc:1034] running on GCE node
I20260812 06:56:20.761957 3292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:20.762836 3292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:20.764106 3292 hybrid_clock.cc:648] HybridClock initialized: now 1786517780764027 us; error 61 us; skew 500 ppm
I20260812 06:56:20.764446 3292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:20.766916 3292 webserver.cc:460] Webserver started at http://127.26.226.3:46657/ using document root <none> and password file <none>
I20260812 06:56:20.767637 3292 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:20.767781 3292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:20.768191 3292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:20.771191 3292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "2b7e0fad29d34010a0274cc123d33d09"
format_stamp: "Formatted at 2026-08-12 06:56:20 on dist-test-slave-t3q3"
server_key: "10a433794755223e5e19f6ef68b991d0"
server_key_iv: "08bc6deecf59ae1094258e5c9a7236db"
server_key_version: "encryptionkey@0"
I20260812 06:56:20.772029 3292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "2b7e0fad29d34010a0274cc123d33d09"
format_stamp: "Formatted at 2026-08-12 06:56:20 on dist-test-slave-t3q3"
server_key: "10a433794755223e5e19f6ef68b991d0"
server_key_iv: "08bc6deecf59ae1094258e5c9a7236db"
server_key_version: "encryptionkey@0"
I20260812 06:56:20.777489 3292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 06:56:20.781059 3313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:20.782478 3292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:20.782665 3292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "2b7e0fad29d34010a0274cc123d33d09"
format_stamp: "Formatted at 2026-08-12 06:56:20 on dist-test-slave-t3q3"
server_key: "10a433794755223e5e19f6ef68b991d0"
server_key_iv: "08bc6deecf59ae1094258e5c9a7236db"
server_key_version: "encryptionkey@0"
I20260812 06:56:20.782908 3292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:20.788918 3292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:20.789740 3292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:20.790021 3292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:20.790987 3292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:20.792467 3292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:20.792588 3292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:20.792733 3292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:20.792825 3292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:20.829274 3292 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:44109
I20260812 06:56:20.829370 3492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:44109 every 8 connection(s)
I20260812 06:56:20.830663 3292 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:20.838668 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 3292
I20260812 06:56:20.838874 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:20.839486 27528 external_mini_cluster.cc:1428] Setting key 3a8e19536d7f08147433dcc54293bbfa
I20260812 06:56:20.839807 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.833548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:44285 (local address 127.26.226.62:44545)
0812 06:56:20.833739 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:20.833744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:20.835403 (+ 1659us) server_negotiation.cc:407] Connection header received
0812 06:56:20.837235 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:20.837240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:20.837298 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:20.837400 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:20.839027 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:20.839032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:20.839036 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:20.839056 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:20.839090 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:20.839103 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:20.839208 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:20.839452 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 06:56:20.839508 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:56:20.841002 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.833990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:44545 (local address 127.26.226.3:44285)
0812 06:56:20.835171 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:20.835268 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:20.836952 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:20.837611 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:20.837620 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:20.837720 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:20.838469 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:20.838473 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:20.838527 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:20.838651 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:20.839216 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:20.839231 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:20.839240 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:56:20.839537 (+ 297us) client_negotiation.cc:240] Negotiation successful
0812 06:56:20.839827 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":275,"threads_started":1}
I20260812 06:56:20.842475 3493 heartbeater.cc:347] Connected to a master server at 127.26.226.62:44545
I20260812 06:56:20.842792 3493 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:20.843389 3493 heartbeater.cc:511] Master 127.26.226.62:44545 requested a full tablet report, sending...
I20260812 06:56:20.844826 2757 ts_manager.cc:194] Registered new tserver with Master: 2b7e0fad29d34010a0274cc123d33d09 (127.26.226.3:44109)
I20260812 06:56:20.855271 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:56:20.862936 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.860269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.26.226.62:44545)
0812 06:56:20.860528 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:20.860550 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:20.860956 (+ 406us) server_negotiation.cc:407] Connection header received
0812 06:56:20.861142 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:20.861146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:20.861205 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:20.861298 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:20.862314 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:20.862318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:20.862322 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:20.862337 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:20.862369 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:20.862382 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:20.862483 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:20.862523 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 06:56:20.862586 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 06:56:20.867527 2757 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:20.874917 2757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:20.902786 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.887362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:36483 (local address 127.0.0.1:38246)
0812 06:56:20.889420 (+ 2058us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:20.889465 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:20.889652 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:20.898284 (+ 8632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:20.898299 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:20.898435 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:20.900201 (+ 1766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:20.900205 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:20.900238 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:20.900337 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:20.901915 (+ 1578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:20.901927 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:20.901938 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 06:56:20.902138 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 06:56:20.902248 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1796,"threads_started":1}
I20260812 06:56:20.904027 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.887584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38246 (local address 127.26.226.2:36483)
0812 06:56:20.888941 (+ 1357us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:20.888951 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:20.889804 (+ 853us) server_negotiation.cc:407] Connection header received
0812 06:56:20.889889 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:20.889898 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:20.896529 (+ 6631us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:20.896790 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:20.901535 (+ 4745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:20.901544 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:20.901560 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:20.901581 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:20.901676 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:20.901698 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:20.901847 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:20.903398 (+ 1551us) server_negotiation.cc:299] Negotiation successful
0812 06:56:20.903515 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1157,"thread_start_us":1031,"threads_started":1}
I20260812 06:56:20.906023 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.888278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.26.226.1:43567)
0812 06:56:20.901338 (+ 13060us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:20.901352 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:20.901394 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:56:20.901479 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:20.901487 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:20.901745 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:20.902009 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:20.905042 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:20.905050 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:20.905064 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:20.905083 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:20.905148 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:20.905168 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:20.905322 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:20.905542 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 06:56:20.905659 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12832,"thread_start_us":12656,"threads_started":1}
I20260812 06:56:20.906049 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.887621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43567 (local address 127.0.0.1:34696)
0812 06:56:20.899383 (+ 11762us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:20.899413 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:20.899572 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:20.902063 (+ 2491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:20.902072 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:20.902112 (+ 40us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:20.903652 (+ 1540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:20.903656 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:20.903672 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:20.903722 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:20.905291 (+ 1569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:20.905296 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:20.905302 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:56:20.905468 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 06:56:20.905527 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11535,"thread_start_us":1357,"threads_started":1}
I20260812 06:56:20.906405 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.890129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:44109 (local address 127.0.0.1:49996)
0812 06:56:20.903260 (+ 13131us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:20.903275 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:20.903412 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:20.904230 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:20.904236 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:20.904258 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:20.904952 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:20.904958 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:20.904975 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:20.905027 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:20.905544 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:20.905550 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:20.905556 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:56:20.905723 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 06:56:20.905775 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13002,"thread_start_us":8775,"threads_started":1}
I20260812 06:56:20.908692 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:20.890452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49996 (local address 127.26.226.3:44109)
0812 06:56:20.902763 (+ 12311us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:20.902775 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:20.903393 (+ 618us) server_negotiation.cc:407] Connection header received
0812 06:56:20.903575 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:20.903586 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:20.903815 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:20.903968 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:20.905154 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:20.905163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:20.905176 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:20.905199 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:20.905273 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:20.905293 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:20.908096 (+ 2803us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:20.908185 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 06:56:20.908285 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11238,"thread_start_us":12640,"threads_started":1}
I20260812 06:56:20.908664 3183 tablet_service.cc:1467] Processing CreateTablet for tablet 81390319cf8549fc886f362aab0e9ffd (DEFAULT_TABLE table=test-table [id=120f676aeea540ac9bf3a2ec8db1f29b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:20.909785 2955 tablet_service.cc:1467] Processing CreateTablet for tablet 81390319cf8549fc886f362aab0e9ffd (DEFAULT_TABLE table=test-table [id=120f676aeea540ac9bf3a2ec8db1f29b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:20.910408 3183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81390319cf8549fc886f362aab0e9ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:20.911266 2955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81390319cf8549fc886f362aab0e9ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:20.912340 3387 tablet_service.cc:1467] Processing CreateTablet for tablet 81390319cf8549fc886f362aab0e9ffd (DEFAULT_TABLE table=test-table [id=120f676aeea540ac9bf3a2ec8db1f29b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:20.914007 3387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81390319cf8549fc886f362aab0e9ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:20.927327 3524 tablet_bootstrap.cc:492] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918: Bootstrap starting.
I20260812 06:56:20.929719 3525 tablet_bootstrap.cc:492] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: Bootstrap starting.
I20260812 06:56:20.930507 3522 tablet_bootstrap.cc:492] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: Bootstrap starting.
I20260812 06:56:20.931715 3083 heartbeater.cc:503] Master 127.26.226.62:44545 was elected leader, sending a full tablet report...
I20260812 06:56:20.932782 3524 tablet_bootstrap.cc:654] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:20.933969 3524 log.cc:824] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:20.934473 3522 tablet_bootstrap.cc:654] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:20.935001 3525 tablet_bootstrap.cc:654] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:20.935922 3522 log.cc:824] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:20.936489 3525 log.cc:824] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:20.938742 3524 tablet_bootstrap.cc:492] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918: No bootstrap required, opened a new log
I20260812 06:56:20.939028 3524 ts_tablet_manager.cc:1397] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260812 06:56:20.939127 3522 tablet_bootstrap.cc:492] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: No bootstrap required, opened a new log
I20260812 06:56:20.939392 3522 ts_tablet_manager.cc:1397] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 06:56:20.939582 3525 tablet_bootstrap.cc:492] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: No bootstrap required, opened a new log
I20260812 06:56:20.939895 3525 ts_tablet_manager.cc:1397] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260812 06:56:20.946539 3524 raft_consensus.cc:348] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 } } peers { permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 } } peers { permanent_uuid: "2b7e0fad29d34010a0274cc123d33d09" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44109 } }
I20260812 06:56:20.946956 3524 raft_consensus.cc:374] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:20.947096 3524 raft_consensus.cc:729] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8edbfb1552024ebb91b6c6f2bdba5918, State: Initialized, Role: FOLLOWER
I20260812 06:56:20.947734 3524 consensus_queue.cc:260] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 } } peers { permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 } } peers { permanent_uuid: "2b7e0fad29d34010a0274cc123d33d09" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44109 } }
I20260812 06:56:20.949570 3524 ts_tablet_manager.cc:1428] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918: Time spent starting tablet: real 0.010s user 0.011s sys 0.001s
I20260812 06:56:20.950016 3522 raft_consensus.cc:348] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 } } peers { permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 } } peers { permanent_uuid: "2b7e0fad29d34010a0274cc123d33d09" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44109 } }
I20260812 06:56:20.950515 3522 raft_consensus.cc:374] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:20.950354 3525 raft_consensus.cc:348] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 } } peers { permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 } } peers { permanent_uuid: "2b7e0fad29d34010a0274cc123d33d09" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44109 } }
I20260812 06:56:20.950645 3522 raft_consensus.cc:729] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93d4730d1f5049ddad9ebec19da393f1, State: Initialized, Role: FOLLOWER
I20260812 06:56:20.950825 3525 raft_consensus.cc:374] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:20.950953 3525 raft_consensus.cc:729] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7e0fad29d34010a0274cc123d33d09, State: Initialized, Role: FOLLOWER
I20260812 06:56:20.951550 3525 consensus_queue.cc:260] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 } } peers { permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 } } peers { permanent_uuid: "2b7e0fad29d34010a0274cc123d33d09" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44109 } }
I20260812 06:56:20.951548 3522 consensus_queue.cc:260] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 } } peers { permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 } } peers { permanent_uuid: "2b7e0fad29d34010a0274cc123d33d09" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44109 } }
I20260812 06:56:20.953500 3493 heartbeater.cc:503] Master 127.26.226.62:44545 was elected leader, sending a full tablet report...
I20260812 06:56:20.953562 3287 heartbeater.cc:503] Master 127.26.226.62:44545 was elected leader, sending a full tablet report...
I20260812 06:56:20.954066 3522 ts_tablet_manager.cc:1428] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: Time spent starting tablet: real 0.015s user 0.009s sys 0.006s
I20260812 06:56:20.954092 3525 ts_tablet_manager.cc:1428] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260812 06:56:21.028342 3538 raft_consensus.cc:482] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:21.028769 3538 raft_consensus.cc:504] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 } } peers { permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 } } peers { permanent_uuid: "2b7e0fad29d34010a0274cc123d33d09" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44109 } }
I20260812 06:56:21.030762 3538 leader_election.cc:288] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93d4730d1f5049ddad9ebec19da393f1 (127.26.226.2:36483), 8edbfb1552024ebb91b6c6f2bdba5918 (127.26.226.1:43567)
I20260812 06:56:21.033773 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.031171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:36483 (local address 127.26.226.3:44399)
0812 06:56:21.031590 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:21.031611 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:21.031734 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:21.032112 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:21.032119 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:21.032145 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:21.032891 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:21.032893 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:21.032912 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:21.032954 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:21.033318 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:21.033323 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:21.033329 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:56:21.033441 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 06:56:21.033504 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267}
I20260812 06:56:21.033793 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.031186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:44399 (local address 127.26.226.2:36483)
0812 06:56:21.031404 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:21.031411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:21.031743 (+ 332us) server_negotiation.cc:407] Connection header received
0812 06:56:21.031868 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:21.031873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:21.031937 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:21.032088 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:21.033135 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:21.033141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:21.033146 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:21.033167 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:21.033205 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:21.033216 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:21.033343 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:21.033434 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 06:56:21.033497 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 06:56:21.034185 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.031469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:33915 (local address 127.26.226.1:43567)
0812 06:56:21.031682 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:21.031688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:21.032278 (+ 590us) server_negotiation.cc:407] Connection header received
0812 06:56:21.032401 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:21.032407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:21.032469 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:21.032547 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:21.033537 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:21.033542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:21.033547 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:21.033565 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:21.033605 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:21.033616 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:21.033749 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:21.033847 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 06:56:21.033910 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 06:56:21.034346 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.031314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43567 (local address 127.26.226.3:33915)
0812 06:56:21.032184 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:21.032207 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:21.032304 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:21.032610 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:21.032616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:21.032643 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:21.033330 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:21.033332 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:21.033346 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:21.033383 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:21.033717 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:21.033722 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:21.033727 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:56:21.033863 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 06:56:21.033910 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":478,"threads_started":1}
I20260812 06:56:21.035254 3212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81390319cf8549fc886f362aab0e9ffd" candidate_uuid: "2b7e0fad29d34010a0274cc123d33d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93d4730d1f5049ddad9ebec19da393f1" is_pre_election: true
I20260812 06:56:21.035511 2994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81390319cf8549fc886f362aab0e9ffd" candidate_uuid: "2b7e0fad29d34010a0274cc123d33d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" is_pre_election: true
I20260812 06:56:21.035782 3212 raft_consensus.cc:2454] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b7e0fad29d34010a0274cc123d33d09 in term 0.
I20260812 06:56:21.036149 2994 raft_consensus.cc:2454] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b7e0fad29d34010a0274cc123d33d09 in term 0.
I20260812 06:56:21.036804 3319 leader_election.cc:302] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2b7e0fad29d34010a0274cc123d33d09, 8edbfb1552024ebb91b6c6f2bdba5918, 93d4730d1f5049ddad9ebec19da393f1; no voters:
I20260812 06:56:21.037252 3538 raft_consensus.cc:2781] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:21.037412 3538 raft_consensus.cc:482] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:21.037531 3538 raft_consensus.cc:3037] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:21.039788 3538 raft_consensus.cc:504] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 } } peers { permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 } } peers { permanent_uuid: "2b7e0fad29d34010a0274cc123d33d09" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44109 } }
I20260812 06:56:21.040508 3538 leader_election.cc:288] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [CANDIDATE]: Term 1 election: Requested vote from peers 93d4730d1f5049ddad9ebec19da393f1 (127.26.226.2:36483), 8edbfb1552024ebb91b6c6f2bdba5918 (127.26.226.1:43567)
I20260812 06:56:21.040962 3212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81390319cf8549fc886f362aab0e9ffd" candidate_uuid: "2b7e0fad29d34010a0274cc123d33d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93d4730d1f5049ddad9ebec19da393f1"
I20260812 06:56:21.041111 2994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81390319cf8549fc886f362aab0e9ffd" candidate_uuid: "2b7e0fad29d34010a0274cc123d33d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8edbfb1552024ebb91b6c6f2bdba5918"
I20260812 06:56:21.041183 3212 raft_consensus.cc:3037] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:21.041365 2994 raft_consensus.cc:3037] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:21.043602 3212 raft_consensus.cc:2454] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b7e0fad29d34010a0274cc123d33d09 in term 1.
I20260812 06:56:21.043666 2994 raft_consensus.cc:2454] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b7e0fad29d34010a0274cc123d33d09 in term 1.
I20260812 06:56:21.044106 3319 leader_election.cc:302] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7e0fad29d34010a0274cc123d33d09, 93d4730d1f5049ddad9ebec19da393f1; no voters:
I20260812 06:56:21.044447 3538 raft_consensus.cc:2781] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:21.044930 3538 raft_consensus.cc:686] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 1 LEADER]: Becoming Leader. State: Replica: 2b7e0fad29d34010a0274cc123d33d09, State: Running, Role: LEADER
I20260812 06:56:21.045420 3538 consensus_queue.cc:237] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 } } peers { permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 } } peers { permanent_uuid: "2b7e0fad29d34010a0274cc123d33d09" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44109 } }
I20260812 06:56:21.050362 2757 catalog_manager.cc:5495] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b7e0fad29d34010a0274cc123d33d09 (127.26.226.3). New cstate: current_term: 1 leader_uuid: "2b7e0fad29d34010a0274cc123d33d09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b7e0fad29d34010a0274cc123d33d09" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44109 } health_report { overall_health: HEALTHY } } }
I20260812 06:56:21.064471 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.062211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50002 (local address 127.26.226.3:44109)
0812 06:56:21.062411 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:21.062418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:21.062438 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:56:21.062688 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:21.062693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:21.062780 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:21.062899 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:21.063794 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:21.063799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:21.063804 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:21.063830 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:21.063863 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:21.063875 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:21.064010 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:21.064144 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 06:56:21.064217 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:56:21.072979 3212 raft_consensus.cc:1261] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1 [term 1 FOLLOWER]: Refusing update from remote peer 2b7e0fad29d34010a0274cc123d33d09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:21.072984 2994 raft_consensus.cc:1261] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918 [term 1 FOLLOWER]: Refusing update from remote peer 2b7e0fad29d34010a0274cc123d33d09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:21.073936 3541 consensus_queue.cc:1035] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8edbfb1552024ebb91b6c6f2bdba5918" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:21.074311 3538 consensus_queue.cc:1035] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93d4730d1f5049ddad9ebec19da393f1" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:56:21.082533 3498 tablet.cc:2369] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:21.086843 3546 mvcc.cc:205] Tried to move back new op lower bound from 7317576831264714752 to 7317576831166070784. Current Snapshot: MvccSnapshot[applied={T|T < 7317576831264714752}]
I20260812 06:56:21.089763 3548 mvcc.cc:205] Tried to move back new op lower bound from 7317576831264714752 to 7317576831166070784. Current Snapshot: MvccSnapshot[applied={T|T < 7317576831264714752}]
I20260812 06:56:21.107388 2757 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:21.107699 2757 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:21.111207 2757 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b96470b0e21946af9f056668ba56b43f: Sending DeleteTablet for 3 replicas of tablet 81390319cf8549fc886f362aab0e9ffd
I20260812 06:56:21.112387 2955 tablet_service.cc:1514] Processing DeleteTablet for tablet 81390319cf8549fc886f362aab0e9ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:21 UTC) from {username='slave'} at 127.0.0.1:34696
I20260812 06:56:21.112812 3387 tablet_service.cc:1514] Processing DeleteTablet for tablet 81390319cf8549fc886f362aab0e9ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:21 UTC) from {username='slave'} at 127.0.0.1:49996
I20260812 06:56:21.113644 3183 tablet_service.cc:1514] Processing DeleteTablet for tablet 81390319cf8549fc886f362aab0e9ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:21 UTC) from {username='slave'} at 127.0.0.1:38246
I20260812 06:56:21.114027 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 2846
I20260812 06:56:21.125212 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.115519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34700 (local address 127.26.226.1:43567)
0812 06:56:21.115691 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:21.115698 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:21.122253 (+ 6555us) server_negotiation.cc:407] Connection header received
0812 06:56:21.122432 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:21.122455 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:21.122566 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:21.122671 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:21.124340 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:21.124346 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:21.124352 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:21.124372 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:21.124414 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:21.124431 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:21.124561 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:21.124818 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 06:56:21.124885 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:56:21.126426 3072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:21.127589 3565 tablet_replica.cc:330] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: stopping tablet replica
I20260812 06:56:21.128293 3566 tablet_replica.cc:330] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: stopping tablet replica
I20260812 06:56:21.128953 3565 raft_consensus.cc:2229] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:21.129562 3566 raft_consensus.cc:2229] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:21.130241 3566 raft_consensus.cc:2258] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:21.131080 3565 raft_consensus.cc:2258] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:21.133837 3566 ts_tablet_manager.cc:1905] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:21.136014 3565 ts_tablet_manager.cc:1905] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:21.144078 3566 ts_tablet_manager.cc:1918] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:21.144352 3566 log.cc:1196] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/81390319cf8549fc886f362aab0e9ffd
I20260812 06:56:21.144984 3566 ts_tablet_manager.cc:1939] T 81390319cf8549fc886f362aab0e9ffd P 93d4730d1f5049ddad9ebec19da393f1: Deleting consensus metadata
I20260812 06:56:21.147066 2737 catalog_manager.cc:4841] TS 93d4730d1f5049ddad9ebec19da393f1 (127.26.226.2:36483): tablet 81390319cf8549fc886f362aab0e9ffd (table test-table [id=120f676aeea540ac9bf3a2ec8db1f29b]) successfully deleted
I20260812 06:56:21.147677 3565 ts_tablet_manager.cc:1918] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:21.147898 3565 log.cc:1196] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/81390319cf8549fc886f362aab0e9ffd
I20260812 06:56:21.148487 3565 ts_tablet_manager.cc:1939] T 81390319cf8549fc886f362aab0e9ffd P 2b7e0fad29d34010a0274cc123d33d09: Deleting consensus metadata
I20260812 06:56:21.149972 2734 catalog_manager.cc:4841] TS 2b7e0fad29d34010a0274cc123d33d09 (127.26.226.3:44109): tablet 81390319cf8549fc886f362aab0e9ffd (table test-table [id=120f676aeea540ac9bf3a2ec8db1f29b]) successfully deleted
I20260812 06:56:21.250257 3563 tablet_replica.cc:330] T 81390319cf8549fc886f362aab0e9ffd P 8edbfb1552024ebb91b6c6f2bdba5918: stopping tablet replica
I20260812 06:56:21.250315 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 2846
W20260812 06:56:21.288254 2736 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.226.1:43567 (error 108)
I20260812 06:56:21.289120 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 3094
I20260812 06:56:21.290580 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.289951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43567 (local address 127.0.0.1:34716)
0812 06:56:21.290248 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:21.290396 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260812 06:56:21.290884 2736 catalog_manager.cc:4584] TS 8edbfb1552024ebb91b6c6f2bdba5918 (127.26.226.1:43567): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81390319cf8549fc886f362aab0e9ffd: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43567: connect: Connection refused (error 111)
I20260812 06:56:21.292742 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.290039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.26.226.2:36483)
0812 06:56:21.290301 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:21.290308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:21.290333 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:56:21.290610 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:21.290618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:21.290748 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:21.290860 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:21.292101 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:21.292108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:21.292116 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:21.292137 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:21.292177 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:21.292194 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:21.292330 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:21.292384 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 06:56:21.292446 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260812 06:56:21.293529 3279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:21.344942 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.344335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43567 (local address 127.0.0.1:34718)
0812 06:56:21.344620 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:21.344723 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260812 06:56:21.346235 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.345811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43567 (local address 127.0.0.1:34730)
0812 06:56:21.346002 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:21.346098 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 06:56:21.383073 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 3094
I20260812 06:56:21.407125 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.406542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43567 (local address 127.0.0.1:34742)
0812 06:56:21.406770 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:21.406917 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 06:56:21.408285 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.407870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43567 (local address 127.0.0.1:34758)
0812 06:56:21.408051 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:21.408172 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 06:56:21.419217 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 3292
I20260812 06:56:21.422505 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.420128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50008 (local address 127.26.226.3:44109)
0812 06:56:21.420342 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:21.420348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:21.420377 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:56:21.420541 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:21.420549 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:21.420623 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:21.420734 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:21.421774 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:21.421781 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:21.421786 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:21.421807 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:21.421852 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:21.421868 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:21.422018 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:21.422104 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 06:56:21.422170 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 06:56:21.423254 3484 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:21.489465 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.488915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43567 (local address 127.0.0.1:34764)
0812 06:56:21.489173 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:21.489291 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260812 06:56:21.490706 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.490292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43567 (local address 127.0.0.1:34778)
0812 06:56:21.490505 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:21.490585 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260812 06:56:21.545513 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 3292
I20260812 06:56:21.576942 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 2712
I20260812 06:56:21.578213 2804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:21.688305 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 2712
2026-08-12T06:56:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2883 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 06:56:21.740271 27528 mini_oidc.cc:152] Starting JWKS server
I20260812 06:56:21.743083 27528 webserver.cc:460] Webserver started at https://127.0.0.1:40907/ using document root <none> and password file <none>
I20260812 06:56:21.743424 27528 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 06:56:21.744208 27528 webserver.cc:460] Webserver started at http://127.0.0.1:44045/ using document root <none> and password file <none>
2026-08-12T06:56:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:21Z Disabled control of system clock
I20260812 06:56:21.781162 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33059
--webserver_interface=127.26.226.62
--webserver_port=0
--builtin_ntp_servers=127.26.226.20:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:33059
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40907/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/testchainca.cert with env {}
W20260812 06:56:22.127097 3591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:22.127406 3591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:22.127497 3591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:22.134864 3591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:22.134982 3591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:22.135021 3591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:22.135077 3591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:22.135123 3591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:56:22.135187 3591 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:56:22.135236 3591 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40907/jwks.json
I20260812 06:56:22.145398 3591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:33059
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40907/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:22.146770 3591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:22.148078 3591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:22.160173 3607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:22.160235 3603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:22.160266 3602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:22.161499 3591 server_base.cc:1034] running on GCE node
I20260812 06:56:22.162340 3591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:22.163800 3591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:22.165030 3591 hybrid_clock.cc:648] HybridClock initialized: now 1786517782164965 us; error 81 us; skew 500 ppm
I20260812 06:56:22.165395 3591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:22.167738 3591 webserver.cc:460] Webserver started at http://127.26.226.62:35673/ using document root <none> and password file <none>
I20260812 06:56:22.168504 3591 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:22.168650 3591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:22.168973 3591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:22.171906 3591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "439a599933e845fbbb68509cf34a147e"
format_stamp: "Formatted at 2026-08-12 06:56:22 on dist-test-slave-t3q3"
server_key: "d483febc8baec8459ff01cfa10460b0f"
server_key_iv: "d084e5db6ee8264222c1ce01a220ecf8"
server_key_version: "encryptionkey@0"
I20260812 06:56:22.172839 3591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "439a599933e845fbbb68509cf34a147e"
format_stamp: "Formatted at 2026-08-12 06:56:22 on dist-test-slave-t3q3"
server_key: "d483febc8baec8459ff01cfa10460b0f"
server_key_iv: "d084e5db6ee8264222c1ce01a220ecf8"
server_key_version: "encryptionkey@0"
I20260812 06:56:22.178001 3591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:56:22.181492 3618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:22.182826 3591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:22.183017 3591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "439a599933e845fbbb68509cf34a147e"
format_stamp: "Formatted at 2026-08-12 06:56:22 on dist-test-slave-t3q3"
server_key: "d483febc8baec8459ff01cfa10460b0f"
server_key_iv: "d084e5db6ee8264222c1ce01a220ecf8"
server_key_version: "encryptionkey@0"
I20260812 06:56:22.183233 3591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:22.196431 3591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:22.197321 3591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:22.197662 3591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:22.217350 3591 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:33059
I20260812 06:56:22.217357 3700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:33059 every 8 connection(s)
I20260812 06:56:22.218844 3591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:22.222946 3701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:22.226176 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 3591
I20260812 06:56:22.226414 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:22.227065 27528 external_mini_cluster.cc:1428] Setting key fea9d496a184e26fb5da36d03a6c2125
I20260812 06:56:22.236213 3701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e: Bootstrap starting.
I20260812 06:56:22.240207 3701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:22.241539 3701 log.cc:824] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:22.242516 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:22.228979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.26.226.62:33059)
0812 06:56:22.230123 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:22.230135 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:22.230180 (+ 45us) server_negotiation.cc:407] Connection header received
0812 06:56:22.232034 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:22.232155 (+ 121us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:22.232710 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:22.233103 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:22.233836 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:22.235429 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:22.237366 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:22.238127 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:22.238282 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:22.239614 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:22.239622 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:22.239636 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:22.240327 (+ 691us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:22.240452 (+ 125us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:22.240475 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:22.240576 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:22.240834 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 06:56:22.241225 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":434,"threads_started":1}
I20260812 06:56:22.244938 3701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e: No bootstrap required, opened a new log
I20260812 06:56:22.247020 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:22.254462 3701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439a599933e845fbbb68509cf34a147e" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33059 } }
I20260812 06:56:22.254860 3701 raft_consensus.cc:374] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:22.254946 3701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 439a599933e845fbbb68509cf34a147e, State: Initialized, Role: FOLLOWER
I20260812 06:56:22.255484 3701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439a599933e845fbbb68509cf34a147e" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33059 } }
I20260812 06:56:22.255689 3701 raft_consensus.cc:388] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:22.255793 3701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:22.255967 3701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:22.258797 3701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439a599933e845fbbb68509cf34a147e" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33059 } }
I20260812 06:56:22.259290 3701 leader_election.cc:302] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 439a599933e845fbbb68509cf34a147e; no voters:
I20260812 06:56:22.260097 3701 leader_election.cc:288] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:22.260308 3716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:22.261246 3716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [term 1 LEADER]: Becoming Leader. State: Replica: 439a599933e845fbbb68509cf34a147e, State: Running, Role: LEADER
I20260812 06:56:22.261888 3716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439a599933e845fbbb68509cf34a147e" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33059 } }
I20260812 06:56:22.262629 3701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:22.265142 3720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 439a599933e845fbbb68509cf34a147e. Latest consensus state: current_term: 1 leader_uuid: "439a599933e845fbbb68509cf34a147e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439a599933e845fbbb68509cf34a147e" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33059 } } }
I20260812 06:56:22.265117 3717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "439a599933e845fbbb68509cf34a147e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439a599933e845fbbb68509cf34a147e" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33059 } } }
I20260812 06:56:22.265648 3720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:22.265648 3717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:22.268117 3729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:22.272861 3729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:22.282671 3729 catalog_manager.cc:1347] Generated new cluster ID: 000796a9379948129a6bf7221ed68461
I20260812 06:56:22.282881 3729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:22.320187 3729 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:22.321621 3729 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:22.332926 3729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 439a599933e845fbbb68509cf34a147e: Generated new TSK 0
I20260812 06:56:22.333842 3729 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 06:56:22.391906 3759 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 06:56:22.392262 3759 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 06:56:22.393019 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:22.362034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51198 (local address 127.26.226.62:33059)
0812 06:56:22.362296 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:22.362303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:22.362730 (+ 427us) server_negotiation.cc:407] Connection header received
0812 06:56:22.363163 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:22.363167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:22.363247 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:22.363368 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:56:22.364123 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:22.364891 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:22.367024 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:22.367474 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:22.367579 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:22.368294 (+ 715us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:56:22.392415 (+ 24121us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40907/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 06:56:22.392771 (+ 356us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51198: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40907/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 06:56:22.393172 3709 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51198: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40907/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 06:56:22.395010 3757 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:22.361907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33059 (local address 127.0.0.1:51198)
0812 06:56:22.362613 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:22.362634 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:22.362837 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:22.363430 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:22.363435 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:22.363449 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:56:22.363863 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:22.363876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:22.365050 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:22.365056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:22.366856 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:22.366873 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:22.367667 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:22.367675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:22.367870 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:22.367947 (+ 77us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:56:22.392744 (+ 24797us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40907/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 06:56:22.393141 (+ 397us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.226.62:33059: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40907/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":344,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.26.226.62:33059: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40907/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 06:56:22.396591 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 3591
I20260812 06:56:22.397812 3694 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:22.497126 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 3591
2026-08-12T06:56:22Z chronyd exiting
I20260812 06:56:22.555153 27528 test_util.cc:175] -----------------------------------------------
I20260812 06:56:22.555341 27528 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517734245872-27528-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (820 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 06:56:22.559216 27528 mini_oidc.cc:152] Starting JWKS server
I20260812 06:56:22.561193 27528 webserver.cc:460] Webserver started at https://127.0.0.1:40605/ using document root <none> and password file <none>
I20260812 06:56:22.561445 27528 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 06:56:22.562193 27528 webserver.cc:460] Webserver started at http://127.0.0.1:38873/ using document root <none> and password file <none>
2026-08-12T06:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:22Z Disabled control of system clock
I20260812 06:56:22.599409 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:46655
--webserver_interface=127.26.226.62
--webserver_port=0
--builtin_ntp_servers=127.26.226.20:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:46655
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40605/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/testchainca.cert with env {}
W20260812 06:56:22.952395 3771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:22.952737 3771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:22.952977 3771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:22.960641 3771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:22.960779 3771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:22.960875 3771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:22.960964 3771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:22.961059 3771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:56:22.961144 3771 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:56:22.961215 3771 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40605/jwks.json
I20260812 06:56:22.972183 3771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:46655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:46655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40605/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:22.973718 3771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:22.975256 3771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:22.988114 3782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:22.988159 3780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:22.988046 3779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:22.989804 3771 server_base.cc:1034] running on GCE node
I20260812 06:56:22.990697 3771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:22.992326 3771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:22.993567 3771 hybrid_clock.cc:648] HybridClock initialized: now 1786517782993523 us; error 64 us; skew 500 ppm
I20260812 06:56:22.993943 3771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:22.996542 3771 webserver.cc:460] Webserver started at http://127.26.226.62:37005/ using document root <none> and password file <none>
I20260812 06:56:22.997288 3771 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:22.997437 3771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:22.997794 3771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:23.000978 3771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "c22e5dade62244bf8ca93f120483791c"
format_stamp: "Formatted at 2026-08-12 06:56:22 on dist-test-slave-t3q3"
server_key: "1a1f32c824a7240625e15cc73eeca05f"
server_key_iv: "d928431907de8cde9b7d41a67d3b12e0"
server_key_version: "encryptionkey@0"
I20260812 06:56:23.001865 3771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "c22e5dade62244bf8ca93f120483791c"
format_stamp: "Formatted at 2026-08-12 06:56:22 on dist-test-slave-t3q3"
server_key: "1a1f32c824a7240625e15cc73eeca05f"
server_key_iv: "d928431907de8cde9b7d41a67d3b12e0"
server_key_version: "encryptionkey@0"
I20260812 06:56:23.007555 3771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:56:23.011346 3790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:23.012904 3771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:23.013149 3771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "c22e5dade62244bf8ca93f120483791c"
format_stamp: "Formatted at 2026-08-12 06:56:22 on dist-test-slave-t3q3"
server_key: "1a1f32c824a7240625e15cc73eeca05f"
server_key_iv: "d928431907de8cde9b7d41a67d3b12e0"
server_key_version: "encryptionkey@0"
I20260812 06:56:23.013419 3771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:23.038688 3771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:23.039671 3771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:23.040048 3771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:23.059999 3771 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:46655
I20260812 06:56:23.060021 3885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:46655 every 8 connection(s)
I20260812 06:56:23.061553 3771 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:23.065637 3888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:23.066733 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 3771
I20260812 06:56:23.066943 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:23.067806 27528 external_mini_cluster.cc:1428] Setting key 303518e20e8d0e2c0fcb76ed14c68a75
I20260812 06:56:23.080435 3888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c: Bootstrap starting.
I20260812 06:56:23.081882 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:23.069882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.26.226.62:46655)
0812 06:56:23.071017 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:23.071029 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:23.071078 (+ 49us) server_negotiation.cc:407] Connection header received
0812 06:56:23.072772 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:23.072795 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:23.073290 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:23.073644 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:23.074229 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.075485 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.077380 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.078082 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.078210 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:23.079482 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:23.079490 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:23.079504 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:23.080048 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:23.080193 (+ 145us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:23.080216 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:23.080306 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:23.080576 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 06:56:23.080941 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":420,"threads_started":1}
I20260812 06:56:23.083824 3888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:23.085122 3888 log.cc:824] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:23.085824 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:23.087906 3888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c: No bootstrap required, opened a new log
I20260812 06:56:23.095108 3888 raft_consensus.cc:348] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22e5dade62244bf8ca93f120483791c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46655 } }
I20260812 06:56:23.095515 3888 raft_consensus.cc:374] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:23.095635 3888 raft_consensus.cc:729] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22e5dade62244bf8ca93f120483791c, State: Initialized, Role: FOLLOWER
I20260812 06:56:23.096302 3888 consensus_queue.cc:260] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22e5dade62244bf8ca93f120483791c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46655 } }
I20260812 06:56:23.096554 3888 raft_consensus.cc:388] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:23.096719 3888 raft_consensus.cc:482] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:23.096897 3888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:23.099149 3888 raft_consensus.cc:504] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22e5dade62244bf8ca93f120483791c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46655 } }
I20260812 06:56:23.099691 3888 leader_election.cc:302] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c22e5dade62244bf8ca93f120483791c; no voters:
I20260812 06:56:23.100358 3888 leader_election.cc:288] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:23.100504 3896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:23.101260 3896 raft_consensus.cc:686] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [term 1 LEADER]: Becoming Leader. State: Replica: c22e5dade62244bf8ca93f120483791c, State: Running, Role: LEADER
I20260812 06:56:23.101871 3896 consensus_queue.cc:237] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22e5dade62244bf8ca93f120483791c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46655 } }
I20260812 06:56:23.102741 3888 sys_catalog.cc:564] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:23.105082 3899 sys_catalog.cc:455] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c22e5dade62244bf8ca93f120483791c. Latest consensus state: current_term: 1 leader_uuid: "c22e5dade62244bf8ca93f120483791c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22e5dade62244bf8ca93f120483791c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46655 } } }
I20260812 06:56:23.105392 3898 sys_catalog.cc:455] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c22e5dade62244bf8ca93f120483791c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22e5dade62244bf8ca93f120483791c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46655 } } }
I20260812 06:56:23.105751 3898 sys_catalog.cc:458] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:23.105628 3899 sys_catalog.cc:458] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:23.107554 3912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:23.113082 3912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:23.121445 3912 catalog_manager.cc:1347] Generated new cluster ID: 1935f543ac9e49a8948fb7b2084da847
I20260812 06:56:23.121568 3912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:23.141865 3912 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:23.143726 3912 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:23.155191 3912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c22e5dade62244bf8ca93f120483791c: Generated new TSK 0
I20260812 06:56:23.156308 3912 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 06:56:23.194559 3937 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 06:56:23.194695 3937 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 06:56:23.195951 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:23.176549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54094 (local address 127.26.226.62:46655)
0812 06:56:23.176796 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:23.176804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:23.177171 (+ 367us) server_negotiation.cc:407] Connection header received
0812 06:56:23.177460 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:23.177464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:23.177534 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:23.177709 (+ 175us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:56:23.178257 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.178982 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.180439 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.180899 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.181039 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:23.181558 (+ 519us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:56:23.195212 (+ 13654us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 06:56:23.195609 (+ 397us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 06:56:23.196146 3935 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:23.176409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:46655 (local address 127.0.0.1:54094)
0812 06:56:23.177059 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:23.177080 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:23.177251 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:23.177732 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:23.177737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:23.177750 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:56:23.178063 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:23.178073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.179137 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.179143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:23.180289 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:23.180302 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.181084 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.181092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:23.181256 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:23.181289 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:56:23.195445 (+ 14156us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 06:56:23.195721 (+ 276us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.226.62:46655: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":416,"threads_started":1}
W20260812 06:56:23.196144 3893 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260812 06:56:23.197566 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 3771
I20260812 06:56:23.198813 3876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:23.293174 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 3771
2026-08-12T06:56:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (799 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 06:56:23.359277 27528 mini_oidc.cc:152] Starting JWKS server
I20260812 06:56:23.361522 27528 webserver.cc:460] Webserver started at https://127.0.0.1:41875/ using document root <none> and password file <none>
I20260812 06:56:23.361819 27528 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 06:56:23.362524 27528 webserver.cc:460] Webserver started at http://127.0.0.1:37363/ using document root <none> and password file <none>
2026-08-12T06:56:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:23Z Disabled control of system clock
I20260812 06:56:23.403532 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34617
--webserver_interface=127.26.226.62
--webserver_port=0
--builtin_ntp_servers=127.26.226.20:41941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:34617
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41875/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 06:56:23.746829 3957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:23.747133 3957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:23.747344 3957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:23.754666 3957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:23.754777 3957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:23.754853 3957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:23.754942 3957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:23.755029 3957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:56:23.755127 3957 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:56:23.755213 3957 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41875/jwks.json
I20260812 06:56:23.765036 3957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:34617
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41875/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:23.766480 3957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:23.768376 3957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:23.780542 3962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:23.780599 3963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:23.780561 3967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:23.781894 3957 server_base.cc:1034] running on GCE node
I20260812 06:56:23.782714 3957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:23.784216 3957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:23.785456 3957 hybrid_clock.cc:648] HybridClock initialized: now 1786517783785412 us; error 56 us; skew 500 ppm
I20260812 06:56:23.785827 3957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:23.791080 3957 webserver.cc:460] Webserver started at http://127.26.226.62:37327/ using document root <none> and password file <none>
I20260812 06:56:23.791851 3957 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:23.791996 3957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:23.792387 3957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:23.795373 3957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "5bcac120dcd946ba9ec68aa6864f9980"
format_stamp: "Formatted at 2026-08-12 06:56:23 on dist-test-slave-t3q3"
server_key: "2aeb8ddaaf136fb892ff42ff42696534"
server_key_iv: "fb430b8fc767fdf5fd4f9d5e291a68f4"
server_key_version: "encryptionkey@0"
I20260812 06:56:23.796291 3957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "5bcac120dcd946ba9ec68aa6864f9980"
format_stamp: "Formatted at 2026-08-12 06:56:23 on dist-test-slave-t3q3"
server_key: "2aeb8ddaaf136fb892ff42ff42696534"
server_key_iv: "fb430b8fc767fdf5fd4f9d5e291a68f4"
server_key_version: "encryptionkey@0"
I20260812 06:56:23.801517 3957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260812 06:56:23.805004 3979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:23.806473 3957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:56:23.806622 3957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "5bcac120dcd946ba9ec68aa6864f9980"
format_stamp: "Formatted at 2026-08-12 06:56:23 on dist-test-slave-t3q3"
server_key: "2aeb8ddaaf136fb892ff42ff42696534"
server_key_iv: "fb430b8fc767fdf5fd4f9d5e291a68f4"
server_key_version: "encryptionkey@0"
I20260812 06:56:23.806862 3957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:23.831480 3957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:23.832362 3957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:23.832666 3957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:23.851516 3957 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:34617
I20260812 06:56:23.851526 4060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:34617 every 8 connection(s)
I20260812 06:56:23.852939 3957 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:23.856947 4063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:23.858084 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 3957
I20260812 06:56:23.858319 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:23.858975 27528 external_mini_cluster.cc:1428] Setting key 00c1a7f085394592b8d568d568434f1e
I20260812 06:56:23.870448 4063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980: Bootstrap starting.
I20260812 06:56:23.874032 4063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:23.874297 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:23.860982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.26.226.62:34617)
0812 06:56:23.862177 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:23.862189 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:23.862234 (+ 45us) server_negotiation.cc:407] Connection header received
0812 06:56:23.864240 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:23.864268 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:23.864897 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:23.865343 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:23.865974 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.867604 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.869213 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.869946 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.870205 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:23.871571 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:23.871579 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:23.871594 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:23.872271 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:23.872368 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:23.872394 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:23.872552 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:23.872804 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 06:56:23.873188 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":417,"threads_started":1}
I20260812 06:56:23.875510 4063 log.cc:824] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:23.878687 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:23.879001 4063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980: No bootstrap required, opened a new log
I20260812 06:56:23.887153 4063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcac120dcd946ba9ec68aa6864f9980" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34617 } }
I20260812 06:56:23.887610 4063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:23.887717 4063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bcac120dcd946ba9ec68aa6864f9980, State: Initialized, Role: FOLLOWER
I20260812 06:56:23.888370 4063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcac120dcd946ba9ec68aa6864f9980" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34617 } }
I20260812 06:56:23.888618 4063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:23.888783 4063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:23.888962 4063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:23.891232 4063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcac120dcd946ba9ec68aa6864f9980" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34617 } }
I20260812 06:56:23.891841 4063 leader_election.cc:302] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bcac120dcd946ba9ec68aa6864f9980; no voters:
I20260812 06:56:23.892575 4063 leader_election.cc:288] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:23.892738 4071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:23.893565 4071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [term 1 LEADER]: Becoming Leader. State: Replica: 5bcac120dcd946ba9ec68aa6864f9980, State: Running, Role: LEADER
I20260812 06:56:23.894172 4071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcac120dcd946ba9ec68aa6864f9980" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34617 } }
I20260812 06:56:23.894904 4063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:23.897336 4072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bcac120dcd946ba9ec68aa6864f9980" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcac120dcd946ba9ec68aa6864f9980" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34617 } } }
I20260812 06:56:23.897857 4072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:23.897755 4075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bcac120dcd946ba9ec68aa6864f9980. Latest consensus state: current_term: 1 leader_uuid: "5bcac120dcd946ba9ec68aa6864f9980" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcac120dcd946ba9ec68aa6864f9980" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34617 } } }
I20260812 06:56:23.898099 4075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:23.905072 4085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:23.910782 4085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:23.920045 4085 catalog_manager.cc:1347] Generated new cluster ID: e8d8b9d5dbd8440d8b4e4a9b8c9e688d
I20260812 06:56:23.920212 4085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:23.941730 4085 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:23.943114 4085 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:23.962678 4085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bcac120dcd946ba9ec68aa6864f9980: Generated new TSK 0
I20260812 06:56:23.963721 4085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:23.987162 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:23.978075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35166 (local address 127.26.226.62:34617)
0812 06:56:23.978298 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:23.978305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:23.978842 (+ 537us) server_negotiation.cc:407] Connection header received
0812 06:56:23.979075 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:23.979081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:23.979177 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:23.979373 (+ 196us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:56:23.979886 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.981064 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.982504 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.983249 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.983381 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:23.983940 (+ 559us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:56:23.986453 (+ 2513us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 06:56:23.986835 (+ 382us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35166: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 06:56:23.987329 4068 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35166: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 06:56:23.988986 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:23.978044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34617 (local address 127.0.0.1:35166)
0812 06:56:23.978734 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:23.978755 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:23.978885 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:23.979408 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:23.979412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:23.979423 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:56:23.979727 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:23.979736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.981246 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.981251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:23.982339 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:23.982352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.983453 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.983457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:23.983601 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:23.983638 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:56:23.986690 (+ 3052us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 06:56:23.988575 (+ 1885us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.226.62:34617: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":1548,"thread_start_us":381,"threads_started":1}
I20260812 06:56:23.990139 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 3957
I20260812 06:56:23.991415 4052 generic_service.cc:196] Checking for leaks (request via RPC)
E20260812 06:56:23.999053 4110 webserver.cc:533] Webserver: sslize failed:
E20260812 06:56:23.999195 4110 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 06:56:24.086158 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 3957
2026-08-12T06:56:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (788 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T06:56:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:24Z Disabled control of system clock
I20260812 06:56:24.185128 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:44685
--webserver_interface=127.26.226.62
--webserver_port=0
--builtin_ntp_servers=127.26.226.20:38913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:44685
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/testchainca.cert with env {}
W20260812 06:56:24.525319 4117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:24.525609 4117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:24.525782 4117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:24.532925 4117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:24.533052 4117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:24.533133 4117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:24.533238 4117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:24.533308 4117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:56:24.533399 4117 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:56:24.533455 4117 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 06:56:24.543259 4117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:44685
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:44685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:24.544572 4117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:24.545872 4117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:24.557785 4126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:24.557833 4123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:24.557905 4124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:24.558534 4117 server_base.cc:1034] running on GCE node
I20260812 06:56:24.559337 4117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:24.560777 4117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:24.561978 4117 hybrid_clock.cc:648] HybridClock initialized: now 1786517784561931 us; error 37 us; skew 500 ppm
I20260812 06:56:24.562327 4117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:24.564587 4117 webserver.cc:460] Webserver started at http://127.26.226.62:36119/ using document root <none> and password file <none>
I20260812 06:56:24.565285 4117 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:24.565409 4117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:24.565725 4117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:24.568815 4117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "d56ec8644ccc431391fcfe8d9fe85e59"
format_stamp: "Formatted at 2026-08-12 06:56:24 on dist-test-slave-t3q3"
server_key: "74d8cef21f33d550b740ef40b7061f60"
server_key_iv: "7f24dd1ede9d0ce323430a8fc9a1d38d"
server_key_version: "encryptionkey@0"
I20260812 06:56:24.569628 4117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "d56ec8644ccc431391fcfe8d9fe85e59"
format_stamp: "Formatted at 2026-08-12 06:56:24 on dist-test-slave-t3q3"
server_key: "74d8cef21f33d550b740ef40b7061f60"
server_key_iv: "7f24dd1ede9d0ce323430a8fc9a1d38d"
server_key_version: "encryptionkey@0"
I20260812 06:56:24.574851 4117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260812 06:56:24.578243 4137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:24.579542 4117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:56:24.579743 4117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "d56ec8644ccc431391fcfe8d9fe85e59"
format_stamp: "Formatted at 2026-08-12 06:56:24 on dist-test-slave-t3q3"
server_key: "74d8cef21f33d550b740ef40b7061f60"
server_key_iv: "7f24dd1ede9d0ce323430a8fc9a1d38d"
server_key_version: "encryptionkey@0"
I20260812 06:56:24.579960 4117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:24.615229 4117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:24.616178 4117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:24.616510 4117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:24.636020 4117 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:44685
I20260812 06:56:24.636090 4233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:44685 every 8 connection(s)
I20260812 06:56:24.637360 4117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:24.639088 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 4117
I20260812 06:56:24.639284 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:24.640184 27528 external_mini_cluster.cc:1428] Setting key 5ef2e4d83519ff7a9d6ac56a9d2c354a
I20260812 06:56:24.642333 4234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:24.654026 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:24.642125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53920 (local address 127.26.226.62:44685)
0812 06:56:24.643109 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:24.643120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:24.643159 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:56:24.644887 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:24.644910 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:24.645359 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:24.645777 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:24.646352 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:24.647674 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:24.649606 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:24.650246 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:24.650376 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:24.651725 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:24.651732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:24.651745 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:24.652360 (+ 615us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:24.652437 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:24.652458 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:24.652551 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:24.652791 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 06:56:24.653145 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":362,"threads_started":1}
I20260812 06:56:24.654940 4234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59: Bootstrap starting.
I20260812 06:56:24.657701 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:24.658316 4234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:24.659320 4234 log.cc:824] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:24.662325 4234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59: No bootstrap required, opened a new log
I20260812 06:56:24.670068 4234 raft_consensus.cc:348] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56ec8644ccc431391fcfe8d9fe85e59" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44685 } }
I20260812 06:56:24.670408 4234 raft_consensus.cc:374] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:24.670496 4234 raft_consensus.cc:729] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d56ec8644ccc431391fcfe8d9fe85e59, State: Initialized, Role: FOLLOWER
I20260812 06:56:24.670966 4234 consensus_queue.cc:260] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56ec8644ccc431391fcfe8d9fe85e59" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44685 } }
I20260812 06:56:24.671175 4234 raft_consensus.cc:388] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:24.671269 4234 raft_consensus.cc:482] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:24.671398 4234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:24.673691 4234 raft_consensus.cc:504] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56ec8644ccc431391fcfe8d9fe85e59" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44685 } }
I20260812 06:56:24.674347 4234 leader_election.cc:302] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d56ec8644ccc431391fcfe8d9fe85e59; no voters:
I20260812 06:56:24.675020 4234 leader_election.cc:288] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:24.675141 4241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:24.675884 4241 raft_consensus.cc:686] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [term 1 LEADER]: Becoming Leader. State: Replica: d56ec8644ccc431391fcfe8d9fe85e59, State: Running, Role: LEADER
I20260812 06:56:24.676443 4241 consensus_queue.cc:237] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56ec8644ccc431391fcfe8d9fe85e59" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44685 } }
I20260812 06:56:24.677264 4234 sys_catalog.cc:564] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:24.679234 4243 sys_catalog.cc:455] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d56ec8644ccc431391fcfe8d9fe85e59. Latest consensus state: current_term: 1 leader_uuid: "d56ec8644ccc431391fcfe8d9fe85e59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56ec8644ccc431391fcfe8d9fe85e59" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44685 } } }
I20260812 06:56:24.679656 4243 sys_catalog.cc:458] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:24.679597 4242 sys_catalog.cc:455] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d56ec8644ccc431391fcfe8d9fe85e59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56ec8644ccc431391fcfe8d9fe85e59" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 44685 } } }
I20260812 06:56:24.680121 4242 sys_catalog.cc:458] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:24.685106 4257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:24.690021 4257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:24.699388 4257 catalog_manager.cc:1347] Generated new cluster ID: 5be1a2d6b9004d18bb352f7ec30b78fd
I20260812 06:56:24.699536 4257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:24.715699 4257 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:24.717195 4257 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:24.733129 4257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d56ec8644ccc431391fcfe8d9fe85e59: Generated new TSK 0
I20260812 06:56:24.734242 4257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:24.793021 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:24.751377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.26.226.62:44685)
0812 06:56:24.751963 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:24.751972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:24.751994 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:56:24.752320 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:24.752324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:24.752391 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:24.752576 (+ 185us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:56:24.753082 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:24.753794 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:24.755203 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:24.755672 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:24.755825 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:24.756378 (+ 553us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:56:24.792211 (+ 35833us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 06:56:24.792651 (+ 440us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53924: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":452}
W20260812 06:56:24.793195 4238 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53924: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 06:56:24.793309 4277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:24.751193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:44685 (local address 127.0.0.1:53924)
0812 06:56:24.751906 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:24.751926 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:24.752114 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:24.752593 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:24.752599 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:24.752611 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:56:24.752903 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:24.752912 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:24.753954 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:24.753960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:24.755042 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:24.755053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:24.755860 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:24.755866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:24.756019 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:24.756052 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:56:24.792540 (+ 36488us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 06:56:24.792853 (+ 313us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.226.62:44685: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":413,"threads_started":1}
I20260812 06:56:24.794956 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 4117
I20260812 06:56:24.796258 4229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:24.893440 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 4117
2026-08-12T06:56:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (789 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (468 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (481 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:25 dist-test-slave-t3q3 krb5kdc[4298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:25 dist-test-slave-t3q3 krb5kdc[4298](info): set up 2 sockets
Aug 12 06:56:25 dist-test-slave-t3q3 krb5kdc[4298](info): commencing operation
krb5kdc: starting...
W20260812 06:56:27.208974 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:56:27 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:56:27.477912 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:36071
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:36071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:27.821552 4316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:27.821918 4316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:27.822098 4316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:27.829582 4316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:27.829748 4316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:27.829871 4316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:27.829978 4316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:27.830068 4316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:27.840490 4316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:36071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:36071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:27.841956 4316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:27.843432 4316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:27.855567 4324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:27.855479 4325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:27.855508 4329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:27.856293 4316 server_base.cc:1034] running on GCE node
I20260812 06:56:27.857064 4316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:27.858422 4316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:27.859695 4316 hybrid_clock.cc:648] HybridClock initialized: now 1786517787859644 us; error 57 us; skew 500 ppm
I20260812 06:56:27.860140 4316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:27 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517787, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:27.863346 4316 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:56:27.865063 4316 webserver.cc:460] Webserver started at http://127.26.226.62:46391/ using document root <none> and password file <none>
I20260812 06:56:27.865849 4316 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:27.866031 4316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:27.866405 4316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:27.870085 4316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "873f9b4e333b454ab4e68c6067cea60d"
format_stamp: "Formatted at 2026-08-12 06:56:27 on dist-test-slave-t3q3"
server_key: "911a5147e54eba162880a741e58ef045"
server_key_iv: "4d151bba65e22cc5ca668d67c50b9b22"
server_key_version: "encryptionkey@0"
I20260812 06:56:27.870994 4316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "873f9b4e333b454ab4e68c6067cea60d"
format_stamp: "Formatted at 2026-08-12 06:56:27 on dist-test-slave-t3q3"
server_key: "911a5147e54eba162880a741e58ef045"
server_key_iv: "4d151bba65e22cc5ca668d67c50b9b22"
server_key_version: "encryptionkey@0"
I20260812 06:56:27.876960 4316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 06:56:27.880733 4345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:27.882131 4316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:27.882442 4316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "873f9b4e333b454ab4e68c6067cea60d"
format_stamp: "Formatted at 2026-08-12 06:56:27 on dist-test-slave-t3q3"
server_key: "911a5147e54eba162880a741e58ef045"
server_key_iv: "4d151bba65e22cc5ca668d67c50b9b22"
server_key_version: "encryptionkey@0"
I20260812 06:56:27.882676 4316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:27.902601 4316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:27.907261 4316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:27.907645 4316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:27.928750 4316 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:36071
I20260812 06:56:27.928774 4430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:36071 every 8 connection(s)
I20260812 06:56:27.930356 4316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:27.934279 4431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:27.934974 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 4316
I20260812 06:56:27.935218 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:27.935943 27528 external_mini_cluster.cc:1428] Setting key bb307b6dcf64903c02aa8d6bcfa4da6f
I20260812 06:56:27.949193 4431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d: Bootstrap starting.
Aug 12 06:56:27 dist-test-slave-t3q3 krb5kdc[4298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:27.952641 4431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:27.953894 4431 log.cc:824] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:27.956883 4431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d: No bootstrap required, opened a new log
I20260812 06:56:27.960487 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:27.938159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.26.226.62:36071)
0812 06:56:27.939017 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:27.939035 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:27.939087 (+ 52us) server_negotiation.cc:407] Connection header received
0812 06:56:27.941137 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:27.941168 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:27.941698 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:27.942168 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:27.943589 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:27.945725 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:27.947061 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:27.948173 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:27.948377 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:27.952911 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:27.952919 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:27.952935 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:27.952991 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:27.956265 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:27.956982 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:27.956993 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:27.957003 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:27.957195 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:27.957601 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:27.957607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:27.957615 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:27.958137 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:27.958468 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:27.958948 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 06:56:27.959382 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":365,"threads_started":1}
I20260812 06:56:27.966252 4431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f9b4e333b454ab4e68c6067cea60d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36071 } }
I20260812 06:56:27.966626 4431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:27.966737 4431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 873f9b4e333b454ab4e68c6067cea60d, State: Initialized, Role: FOLLOWER
I20260812 06:56:27.967258 4431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f9b4e333b454ab4e68c6067cea60d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36071 } }
I20260812 06:56:27.967510 4431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:27.967657 4431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:27.967835 4431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:27.970055 4431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f9b4e333b454ab4e68c6067cea60d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36071 } }
I20260812 06:56:27.970558 4431 leader_election.cc:302] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 873f9b4e333b454ab4e68c6067cea60d; no voters:
I20260812 06:56:27.971166 4431 leader_election.cc:288] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:27.971310 4440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:27.972083 4440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [term 1 LEADER]: Becoming Leader. State: Replica: 873f9b4e333b454ab4e68c6067cea60d, State: Running, Role: LEADER
I20260812 06:56:27.972738 4440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f9b4e333b454ab4e68c6067cea60d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36071 } }
I20260812 06:56:27.973465 4431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:27.976257 4443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 873f9b4e333b454ab4e68c6067cea60d. Latest consensus state: current_term: 1 leader_uuid: "873f9b4e333b454ab4e68c6067cea60d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f9b4e333b454ab4e68c6067cea60d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36071 } } }
I20260812 06:56:27.976583 4443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:27.975853 4442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "873f9b4e333b454ab4e68c6067cea60d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f9b4e333b454ab4e68c6067cea60d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36071 } } }
I20260812 06:56:27.977098 4442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:27.980803 4450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:27.987260 4450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:27.996693 4450 catalog_manager.cc:1347] Generated new cluster ID: 5759fd4cc17e4cd29186882a58a64f9d
I20260812 06:56:27.996832 4450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:28.006166 4450 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:28.007543 4450 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:28.020860 4450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 873f9b4e333b454ab4e68c6067cea60d: Generated new TSK 0
I20260812 06:56:28.021800 4450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:56:28.087433 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36071
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:28.438786 4476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:28.439095 4476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:28.439270 4476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:28.446707 4476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:28.446816 4476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:28.447037 4476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:28.456933 4476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:28.458273 4476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:28.459703 4476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:28.473658 4488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:28.473899 4484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:28.473681 4485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:28.474411 4476 server_base.cc:1034] running on GCE node
I20260812 06:56:28.475018 4476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:28.475991 4476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:28.477254 4476 hybrid_clock.cc:648] HybridClock initialized: now 1786517788477180 us; error 73 us; skew 500 ppm
I20260812 06:56:28.477595 4476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:28 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517788, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:28.480861 4476 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:56:28.482292 4476 webserver.cc:460] Webserver started at http://127.26.226.1:37321/ using document root <none> and password file <none>
I20260812 06:56:28.482995 4476 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:28.483168 4476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:28.483521 4476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:28.486644 4476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "ae15b1d0e23143778fe1fef2f47f6a4a"
format_stamp: "Formatted at 2026-08-12 06:56:28 on dist-test-slave-t3q3"
server_key: "831862aa231a9f9a85a45b2ab994a032"
server_key_iv: "3618046025921aed7b0dce38fc4a88f0"
server_key_version: "encryptionkey@0"
I20260812 06:56:28.487550 4476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "ae15b1d0e23143778fe1fef2f47f6a4a"
format_stamp: "Formatted at 2026-08-12 06:56:28 on dist-test-slave-t3q3"
server_key: "831862aa231a9f9a85a45b2ab994a032"
server_key_iv: "3618046025921aed7b0dce38fc4a88f0"
server_key_version: "encryptionkey@0"
I20260812 06:56:28.492926 4476 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 06:56:28.496479 4498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:28.497812 4476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 06:56:28.497990 4476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "ae15b1d0e23143778fe1fef2f47f6a4a"
format_stamp: "Formatted at 2026-08-12 06:56:28 on dist-test-slave-t3q3"
server_key: "831862aa231a9f9a85a45b2ab994a032"
server_key_iv: "3618046025921aed7b0dce38fc4a88f0"
server_key_version: "encryptionkey@0"
I20260812 06:56:28.498222 4476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:28.512560 4476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:28.516649 4476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:28.516954 4476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:28.517939 4476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:28.519284 4476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:28.519390 4476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:28.519550 4476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:28.519626 4476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:28.553743 4476 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:40999
I20260812 06:56:28.553788 4677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:40999 every 8 connection(s)
I20260812 06:56:28.555207 4476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:28.565289 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 4476
I20260812 06:56:28.565500 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:28.566161 27528 external_mini_cluster.cc:1428] Setting key a93248800930b5b0af8e710093be8a18
Aug 12 06:56:28 dist-test-slave-t3q3 krb5kdc[4298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517788, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:28.576663 4681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:28.557985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:59853 (local address 127.26.226.62:36071)
0812 06:56:28.558446 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:28.558451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:28.559437 (+ 986us) server_negotiation.cc:407] Connection header received
0812 06:56:28.561377 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:28.561382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:28.561478 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:28.561612 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:28.563184 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:28.563935 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:28.565571 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:28.566031 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:28.566179 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:28.570353 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:28.570358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:28.570361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:28.570394 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:28.573188 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:28.574032 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:28.574038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:28.574041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:28.574206 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:28.574664 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:28.574669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:28.574673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:28.575016 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:28.575176 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:28.575951 (+ 775us) server_negotiation.cc:299] Negotiation successful
0812 06:56:28.576267 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":228,"threads_started":1}
I20260812 06:56:28.577865 4682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:28.558550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36071 (local address 127.26.226.1:59853)
0812 06:56:28.559278 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:28.559317 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:28.561093 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:28.561864 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:28.561872 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:28.562411 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:28.562991 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:28.563008 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:28.564141 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:28.564145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:28.565401 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:28.565419 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:28.566270 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:28.566279 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:28.566540 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:28.567314 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:28.567337 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:28.570139 (+ 2802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:28.573335 (+ 3196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:28.573344 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:28.573361 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:28.573901 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:28.574330 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:28.574335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:28.574338 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:28.574539 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:28.575151 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:28.575159 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:28.575525 (+ 366us) client_negotiation.cc:769] Sending connection context
0812 06:56:28.575839 (+ 314us) client_negotiation.cc:240] Negotiation successful
0812 06:56:28.576465 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":292,"threads_started":1}
I20260812 06:56:28.579656 4678 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36071
I20260812 06:56:28.580013 4678 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:28.580960 4678 heartbeater.cc:511] Master 127.26.226.62:36071 requested a full tablet report, sending...
I20260812 06:56:28.583526 4368 ts_manager.cc:194] Registered new tserver with Master: ae15b1d0e23143778fe1fef2f47f6a4a (127.26.226.1:40999)
I20260812 06:56:28.585813 4368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:59853
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:56:28.629125 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36071
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:28.989074 4685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:28.989354 4685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:28.989449 4685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:28.996567 4685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:28.996670 4685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:28.996789 4685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:29.006462 4685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:29.007714 4685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:29.009076 4685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:29.022162 4698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:29.022117 4696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:29.022207 4695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:29.023299 4685 server_base.cc:1034] running on GCE node
I20260812 06:56:29.023977 4685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:29.024959 4685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:29.026186 4685 hybrid_clock.cc:648] HybridClock initialized: now 1786517789026120 us; error 53 us; skew 500 ppm
I20260812 06:56:29.026535 4685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:29.033736 4685 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:56:29.035197 4685 webserver.cc:460] Webserver started at http://127.26.226.2:33259/ using document root <none> and password file <none>
I20260812 06:56:29.035893 4685 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:29.036104 4685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:29.036458 4685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:29.039419 4685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "9b7ebc51e65f4d6781697a206675f318"
format_stamp: "Formatted at 2026-08-12 06:56:29 on dist-test-slave-t3q3"
server_key: "1766638519ac8bacd11f361569885163"
server_key_iv: "f3bfdc21e04741b022e0cb2779efb77a"
server_key_version: "encryptionkey@0"
I20260812 06:56:29.040310 4685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "9b7ebc51e65f4d6781697a206675f318"
format_stamp: "Formatted at 2026-08-12 06:56:29 on dist-test-slave-t3q3"
server_key: "1766638519ac8bacd11f361569885163"
server_key_iv: "f3bfdc21e04741b022e0cb2779efb77a"
server_key_version: "encryptionkey@0"
I20260812 06:56:29.045632 4685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 06:56:29.049108 4713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:29.050406 4685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:56:29.050604 4685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "9b7ebc51e65f4d6781697a206675f318"
format_stamp: "Formatted at 2026-08-12 06:56:29 on dist-test-slave-t3q3"
server_key: "1766638519ac8bacd11f361569885163"
server_key_iv: "f3bfdc21e04741b022e0cb2779efb77a"
server_key_version: "encryptionkey@0"
I20260812 06:56:29.050835 4685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:29.064927 4685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:29.069000 4685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:29.069303 4685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:29.070298 4685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:29.071640 4685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:29.071758 4685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:29.071908 4685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:29.071990 4685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:29.107952 4685 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:36427
I20260812 06:56:29.108001 4893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:36427 every 8 connection(s)
I20260812 06:56:29.109328 4685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:29.116500 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 4685
I20260812 06:56:29.116685 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:29.117275 27528 external_mini_cluster.cc:1428] Setting key 3d4c49af3386a186fb351c3f43a27b49
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:29.131896 4896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.111980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:51943 (local address 127.26.226.62:36071)
0812 06:56:29.112447 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.112452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.113477 (+ 1025us) server_negotiation.cc:407] Connection header received
0812 06:56:29.115386 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.115392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.115484 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.115614 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.117800 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.118800 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.121217 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.121637 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.121751 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.126036 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:29.126041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:29.126045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:29.126076 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:29.128788 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.129534 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.129544 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.129547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.129669 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.130127 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.130136 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.130138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.130408 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:29.130571 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:29.131283 (+ 712us) server_negotiation.cc:299] Negotiation successful
0812 06:56:29.131507 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":234,"threads_started":1}
I20260812 06:56:29.132875 4897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.112572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36071 (local address 127.26.226.2:51943)
0812 06:56:29.113310 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:29.113348 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:29.115078 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:29.115874 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:29.115883 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:29.116739 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:29.117596 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:29.117624 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.118965 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.118969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:29.121032 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:29.121053 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.121885 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.121895 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:29.122224 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.122992 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:29.123026 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:29.125780 (+ 2754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:29.128942 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:29.128951 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:29.128966 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:29.129395 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:29.129790 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:29.129795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:29.129798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:29.129996 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:29.130546 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:29.130554 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:29.130900 (+ 346us) client_negotiation.cc:769] Sending connection context
0812 06:56:29.131167 (+ 267us) client_negotiation.cc:240] Negotiation successful
0812 06:56:29.131719 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":303,"threads_started":1}
I20260812 06:56:29.134330 4894 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36071
I20260812 06:56:29.134685 4894 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:29.135648 4894 heartbeater.cc:511] Master 127.26.226.62:36071 requested a full tablet report, sending...
I20260812 06:56:29.137472 4368 ts_manager.cc:194] Registered new tserver with Master: 9b7ebc51e65f4d6781697a206675f318 (127.26.226.2:36427)
I20260812 06:56:29.138365 4368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:51943
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:56:29.179617 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36071
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:43043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:29.529958 4900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:29.530321 4900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:29.530505 4900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:29.538035 4900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:29.538200 4900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:29.538389 4900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:29.548601 4900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:43043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:29.550038 4900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:29.551427 4900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:29.565784 4908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:29.565832 4911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:29.565742 4913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:29.569507 4900 server_base.cc:1034] running on GCE node
I20260812 06:56:29.570147 4900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:29.571156 4900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:29.572382 4900 hybrid_clock.cc:648] HybridClock initialized: now 1786517789572336 us; error 33 us; skew 500 ppm
I20260812 06:56:29.572675 4900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:29.576407 4900 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:56:29.577889 4900 webserver.cc:460] Webserver started at http://127.26.226.3:42475/ using document root <none> and password file <none>
I20260812 06:56:29.578562 4900 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:29.578678 4900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:29.578950 4900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:29.582017 4900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "9775e080632f497da877b15c9e284e4f"
format_stamp: "Formatted at 2026-08-12 06:56:29 on dist-test-slave-t3q3"
server_key: "4717d79a0b90ebfc4e5c381eec549de0"
server_key_iv: "b72433bb2f47a27300d5e9ee65be9679"
server_key_version: "encryptionkey@0"
I20260812 06:56:29.582748 4900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "9775e080632f497da877b15c9e284e4f"
format_stamp: "Formatted at 2026-08-12 06:56:29 on dist-test-slave-t3q3"
server_key: "4717d79a0b90ebfc4e5c381eec549de0"
server_key_iv: "b72433bb2f47a27300d5e9ee65be9679"
server_key_version: "encryptionkey@0"
I20260812 06:56:29.588236 4900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 06:56:29.589880 4678 heartbeater.cc:503] Master 127.26.226.62:36071 was elected leader, sending a full tablet report...
I20260812 06:56:29.592046 4922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:29.593396 4900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:29.593582 4900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "9775e080632f497da877b15c9e284e4f"
format_stamp: "Formatted at 2026-08-12 06:56:29 on dist-test-slave-t3q3"
server_key: "4717d79a0b90ebfc4e5c381eec549de0"
server_key_iv: "b72433bb2f47a27300d5e9ee65be9679"
server_key_version: "encryptionkey@0"
I20260812 06:56:29.593786 4900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:29.614887 4900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:29.619768 4900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:29.620148 4900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:29.621170 4900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:29.622550 4900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:29.622665 4900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:29.622816 4900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:29.622890 4900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:29.657964 4900 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:38145
I20260812 06:56:29.658027 5093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:38145 every 8 connection(s)
I20260812 06:56:29.659322 4900 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:29.667059 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 4900
I20260812 06:56:29.667276 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:29.667768 27528 external_mini_cluster.cc:1428] Setting key 6d3dfdb021bac1d664761234c67eb7ca
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:29.680154 5099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.661997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:45971 (local address 127.26.226.62:36071)
0812 06:56:29.662522 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.662526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.663354 (+ 828us) server_negotiation.cc:407] Connection header received
0812 06:56:29.665319 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.665325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.665419 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.665542 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.667121 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.668168 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.669780 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.670392 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.670533 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.674171 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:29.674176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:29.674179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:29.674228 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:29.676938 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.677758 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.677762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.677764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.677888 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.678337 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.678340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.678342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.678599 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:29.678725 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:29.679599 (+ 874us) server_negotiation.cc:299] Negotiation successful
0812 06:56:29.679784 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":288,"threads_started":1}
I20260812 06:56:29.681119 5100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.662464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36071 (local address 127.26.226.3:45971)
0812 06:56:29.663185 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:29.663228 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:29.665036 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:29.665796 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:29.665804 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:29.666331 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:29.666931 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:29.666949 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.668340 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.668344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:29.669580 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:29.669592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.670642 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.670652 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:29.670926 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.671553 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:29.671577 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:29.673954 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:29.677135 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:29.677144 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:29.677159 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:29.677639 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:29.678007 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:29.678010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:29.678013 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:29.678238 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:29.678735 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:29.678742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:29.679123 (+ 381us) client_negotiation.cc:769] Sending connection context
0812 06:56:29.679371 (+ 248us) client_negotiation.cc:240] Negotiation successful
0812 06:56:29.679869 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":288,"threads_started":1}
I20260812 06:56:29.682687 5096 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36071
I20260812 06:56:29.683029 5096 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:29.683954 5096 heartbeater.cc:511] Master 127.26.226.62:36071 requested a full tablet report, sending...
I20260812 06:56:29.685760 4367 ts_manager.cc:194] Registered new tserver with Master: 9775e080632f497da877b15c9e284e4f (127.26.226.3:38145)
I20260812 06:56:29.686911 4367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:45971
I20260812 06:56:29.696200 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/test-admin.keytab.
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:29.731678 27528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 06:56:29.737277 5106 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:29.734464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58376 (local address 127.26.226.1:40999)
0812 06:56:29.735173 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.735180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.735223 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:56:29.735302 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.735309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.735526 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.735666 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.736782 (+ 1116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58376: BlockingRecv error: recv got EOF from 127.0.0.1:58376 (error 108)
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":393,"threads_started":1}
W20260812 06:56:29.737233 27528 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5cc' of type FILE
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:29.738934 27528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:29.740399 27528 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:56:29.756466 5106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.742198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.26.226.1:40999)
0812 06:56:29.742392 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.742397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.742469 (+ 72us) server_negotiation.cc:407] Connection header received
0812 06:56:29.742637 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.742640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.742700 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.742786 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.743744 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.744814 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.746143 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.746729 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.746861 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.750558 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:29.750571 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:29.750588 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:29.750632 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:29.753463 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.754234 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.754242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.754244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.754375 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.754819 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.754823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.754825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.755257 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:29.755498 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:29.755859 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 06:56:29.756147 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260812 06:56:29.763130 5106 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:29.761119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58402 (local address 127.26.226.1:40999)
0812 06:56:29.761354 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.761361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.761580 (+ 219us) server_negotiation.cc:407] Connection header received
0812 06:56:29.761795 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.761801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.761884 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.761999 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.762856 (+ 857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58402: BlockingRecv error: recv got EOF from 127.0.0.1:58402 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260812 06:56:29.764726 27528 init.cc:244] Error while opening credential cache '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5cc)
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:29.766336 27528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:29.767647 27528 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:56:29.781921 5106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.768959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58406 (local address 127.26.226.1:40999)
0812 06:56:29.769186 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.769192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.769406 (+ 214us) server_negotiation.cc:407] Connection header received
0812 06:56:29.769540 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.769543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.769600 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.769682 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.770584 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.771312 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.772537 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.772980 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.773101 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.776800 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:29.776805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:29.776809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:29.776842 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:29.779385 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.780174 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.780183 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.780186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.780316 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.780661 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.780664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.780666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.780954 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:29.781162 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:29.781420 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 06:56:29.781638 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260812 06:56:29.786550 5106 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:29.784859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58420 (local address 127.26.226.1:40999)
0812 06:56:29.785070 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.785076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.785266 (+ 190us) server_negotiation.cc:407] Connection header received
0812 06:56:29.785430 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.785435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.785511 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.785619 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.786331 (+ 712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58420: BlockingRecv error: recv got EOF from 127.0.0.1:58420 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 06:56:29.786824 27528 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517734245872-27528-0/krb5kdc/krb5cc' of type FILE
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:29.788583 27528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:29.789891 27528 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:56:29.804620 5106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.791171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58428 (local address 127.26.226.1:40999)
0812 06:56:29.791401 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.791406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.791650 (+ 244us) server_negotiation.cc:407] Connection header received
0812 06:56:29.791773 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.791776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.791829 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.791908 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.792874 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.793588 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.794813 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.795225 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.795320 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.798909 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:29.798914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:29.798918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:29.798948 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:29.801707 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.802499 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.802503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.802506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.802749 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.803145 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.803150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.803152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.803465 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:29.803720 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:29.803988 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 06:56:29.804290 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 06:56:29.806213 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 4476
I20260812 06:56:29.818413 5106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.806910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58432 (local address 127.26.226.1:40999)
0812 06:56:29.807049 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.807053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.807471 (+ 418us) server_negotiation.cc:407] Connection header received
0812 06:56:29.807577 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.807581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.807632 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.807722 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.808665 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.809357 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.810464 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.810887 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.810982 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.812980 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:29.812985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:29.812989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:29.813025 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:29.815849 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.816541 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.816545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.816548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.816691 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.817047 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.817050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.817052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.817323 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:29.817553 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:29.817890 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 06:56:29.818143 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:56:29.819015 4669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:29.903468 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 4476
I20260812 06:56:29.925892 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 4685
Aug 12 06:56:29 dist-test-slave-t3q3 krb5kdc[4298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:56:29.943029 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.926792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.26.226.2:36427)
0812 06:56:29.927358 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.927365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.927391 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:56:29.927453 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.927460 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.927671 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.927799 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.929092 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.930127 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.931927 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.932557 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.932659 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.937407 (+ 4748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:29.937420 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:29.937436 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:29.937471 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:29.940183 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.940853 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.940867 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.940869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.940992 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:29.941395 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:29.941401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:29.941405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:29.941763 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:29.942100 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:29.942433 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 06:56:29.942727 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":300,"threads_started":1}
I20260812 06:56:29.944612 4887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:30.029031 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 4685
I20260812 06:56:30.057276 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 4900
Aug 12 06:56:30 dist-test-slave-t3q3 krb5kdc[4298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:56:30.073236 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:30.058652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.26.226.3:38145)
0812 06:56:30.059377 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:30.059385 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:30.059427 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:56:30.059502 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:30.059509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:30.059720 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:30.059891 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:30.061071 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:30.062098 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:30.063281 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:30.063864 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:30.063965 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:30.067878 (+ 3913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:30.067889 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:30.067903 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:30.067939 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:30.070663 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:30.071327 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:30.071334 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:30.071337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:30.071468 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:30.071833 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:30.071836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:30.071838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:30.072251 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:30.072498 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:30.072723 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 06:56:30.072923 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":378,"threads_started":1}
I20260812 06:56:30.074541 5086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:30.157598 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 4900
I20260812 06:56:30.187297 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 4316
I20260812 06:56:30.188649 4425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:30.293469 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 4316
2026-08-12T06:56:30Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4458 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:30 dist-test-slave-t3q3 krb5kdc[5136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:30 dist-test-slave-t3q3 krb5kdc[5136](info): set up 2 sockets
Aug 12 06:56:30 dist-test-slave-t3q3 krb5kdc[5136](info): commencing operation
krb5kdc: starting...
W20260812 06:56:31.652446 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:56:31 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.62.keytab.
Entry for principal oryx/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.62.keytab.
I20260812 06:56:31.929920 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:36667
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.62.keytab
--principal=oryx/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:36667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:32.268879 5153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:32.269286 5153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:32.269466 5153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:32.276759 5153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:32.276930 5153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:32.277046 5153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:32.277166 5153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:32.277253 5153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:32.287463 5153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:36667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.62.keytab
--principal=oryx/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:36667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:32.288924 5153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:32.290290 5153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:32.303033 5162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:32.302989 5160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:32.303128 5164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:32.303897 5153 server_base.cc:1034] running on GCE node
I20260812 06:56:32.304915 5153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:32.306528 5153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:32.307760 5153 hybrid_clock.cc:648] HybridClock initialized: now 1786517792307712 us; error 40 us; skew 500 ppm
I20260812 06:56:32.308223 5153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:32 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517792, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:32.311681 5153 init.cc:377] Logged in from keytab as oryx/127.26.226.62@KRBTEST.COM (short username oryx)
I20260812 06:56:32.313369 5153 webserver.cc:460] Webserver started at http://127.26.226.62:42221/ using document root <none> and password file <none>
I20260812 06:56:32.314105 5153 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:32.314220 5153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:32.314489 5153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:32.317792 5153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "f4a6c926fc984692809b846da0948692"
format_stamp: "Formatted at 2026-08-12 06:56:32 on dist-test-slave-t3q3"
server_key: "b5c0f3ae21b9f841c485c365bd0b89c4"
server_key_iv: "35daaca376272a01a3642763eb94e1b8"
server_key_version: "encryptionkey@0"
I20260812 06:56:32.318601 5153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "f4a6c926fc984692809b846da0948692"
format_stamp: "Formatted at 2026-08-12 06:56:32 on dist-test-slave-t3q3"
server_key: "b5c0f3ae21b9f841c485c365bd0b89c4"
server_key_iv: "35daaca376272a01a3642763eb94e1b8"
server_key_version: "encryptionkey@0"
I20260812 06:56:32.324222 5153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260812 06:56:32.327724 5178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:32.329180 5153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:56:32.329396 5153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "f4a6c926fc984692809b846da0948692"
format_stamp: "Formatted at 2026-08-12 06:56:32 on dist-test-slave-t3q3"
server_key: "b5c0f3ae21b9f841c485c365bd0b89c4"
server_key_iv: "35daaca376272a01a3642763eb94e1b8"
server_key_version: "encryptionkey@0"
I20260812 06:56:32.329597 5153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:32.342257 5153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:32.346973 5153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:32.347297 5153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:32.366168 5153 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:36667
I20260812 06:56:32.366187 5268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:36667 every 8 connection(s)
I20260812 06:56:32.367686 5153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:32.371516 5269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:32.376327 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 5153
I20260812 06:56:32.376554 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:32.377441 27528 external_mini_cluster.cc:1428] Setting key 9fead9840b93d26beeafe94f9721a3ee
I20260812 06:56:32.383893 5269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692: Bootstrap starting.
I20260812 06:56:32.387818 5269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:32.389034 5269 log.cc:824] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:32.391808 5269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692: No bootstrap required, opened a new log
Aug 12 06:56:32 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:32.401046 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:32.379677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.26.226.62:36667)
0812 06:56:32.380552 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:32.380567 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:32.380622 (+ 55us) server_negotiation.cc:407] Connection header received
0812 06:56:32.382393 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:32.382418 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:32.382830 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:32.383177 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:32.385110 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:32.386610 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:32.388408 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:32.389091 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:32.389288 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:32.393662 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:32.393673 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:32.393687 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:32.393731 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:32.397036 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:32.397839 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:32.397845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:32.397850 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:32.398016 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:32.398414 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:32.398417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:32.398419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:32.398954 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:32.399190 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:32.399641 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 06:56:32.400018 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":431,"threads_started":1}
I20260812 06:56:32.400985 5269 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } }
I20260812 06:56:32.401441 5269 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:32.401571 5269 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4a6c926fc984692809b846da0948692, State: Initialized, Role: FOLLOWER
I20260812 06:56:32.402201 5269 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } }
I20260812 06:56:32.402451 5269 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:32.402626 5269 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:32.402812 5269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:32.405151 5269 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } }
I20260812 06:56:32.405632 5269 leader_election.cc:302] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4a6c926fc984692809b846da0948692; no voters:
I20260812 06:56:32.406322 5269 leader_election.cc:288] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:32.406446 5274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:32.407303 5274 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 1 LEADER]: Becoming Leader. State: Replica: f4a6c926fc984692809b846da0948692, State: Running, Role: LEADER
I20260812 06:56:32.408028 5274 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } }
I20260812 06:56:32.408864 5269 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:32.411128 5275 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } }
I20260812 06:56:32.411454 5275 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:32.411949 5277 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4a6c926fc984692809b846da0948692. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } }
I20260812 06:56:32.412326 5277 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:32.412501 5283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:32.418231 5283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:32.427524 5283 catalog_manager.cc:1347] Generated new cluster ID: 993356870845481c94b7918d6e35d6eb
I20260812 06:56:32.427639 5283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:32.451097 5283 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:32.453118 5283 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:32.462828 5283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692: Generated new TSK 0
I20260812 06:56:32.463919 5283 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab.
Entry for principal oryx/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab.
I20260812 06:56:32.527948 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36667
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab
--principal=oryx/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:32.878443 5303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:32.878729 5303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:32.878826 5303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:32.886018 5303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:32.886129 5303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:32.886245 5303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:32.896598 5303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab
--principal=oryx/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:32.898051 5303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:32.899366 5303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:32.912643 5308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:32.912839 5311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:32.912673 5309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:32.917536 5303 server_base.cc:1034] running on GCE node
I20260812 06:56:32.918190 5303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:32.919183 5303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:32.920416 5303 hybrid_clock.cc:648] HybridClock initialized: now 1786517792920371 us; error 37 us; skew 500 ppm
I20260812 06:56:32.920742 5303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:32 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517792, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:32.924052 5303 init.cc:377] Logged in from keytab as oryx/127.26.226.1@KRBTEST.COM (short username oryx)
I20260812 06:56:32.925550 5303 webserver.cc:460] Webserver started at http://127.26.226.1:44991/ using document root <none> and password file <none>
I20260812 06:56:32.926276 5303 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:32.926424 5303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:32.926764 5303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:32.929746 5303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174"
format_stamp: "Formatted at 2026-08-12 06:56:32 on dist-test-slave-t3q3"
server_key: "3bc34aa9c50cc6034b984a6584862dd6"
server_key_iv: "4506458595526437675a65f9f34100ce"
server_key_version: "encryptionkey@0"
I20260812 06:56:32.930567 5303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174"
format_stamp: "Formatted at 2026-08-12 06:56:32 on dist-test-slave-t3q3"
server_key: "3bc34aa9c50cc6034b984a6584862dd6"
server_key_iv: "4506458595526437675a65f9f34100ce"
server_key_version: "encryptionkey@0"
I20260812 06:56:32.935837 5303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 06:56:32.939314 5320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:32.940683 5303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:32.940873 5303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174"
format_stamp: "Formatted at 2026-08-12 06:56:32 on dist-test-slave-t3q3"
server_key: "3bc34aa9c50cc6034b984a6584862dd6"
server_key_iv: "4506458595526437675a65f9f34100ce"
server_key_version: "encryptionkey@0"
I20260812 06:56:32.941080 5303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:32.956503 5303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:32.960937 5303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:32.961252 5303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:32.962360 5303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:32.963820 5303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:32.963923 5303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:32.964098 5303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:32.964183 5303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260812 06:56:32.999378 5303 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:42641
I20260812 06:56:32.999439 5506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:42641 every 8 connection(s)
I20260812 06:56:33.000732 5303 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:33.004244 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 5303
I20260812 06:56:33.004485 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:33.005507 27528 external_mini_cluster.cc:1428] Setting key 11e96083ef26ec2961b2604faeac07fc
Aug 12 06:56:33 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517792, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:33.026948 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:33.003472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:49613 (local address 127.26.226.62:36667)
0812 06:56:33.004041 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:33.004048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:33.005376 (+ 1328us) server_negotiation.cc:407] Connection header received
0812 06:56:33.007947 (+ 2571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:33.007951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:33.008017 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:33.008170 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:33.010630 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.011355 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.014403 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.014888 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.014995 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:33.019643 (+ 4648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:33.019652 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:33.019660 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:33.019705 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:33.023405 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:33.024213 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:33.024222 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:33.024228 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:33.024392 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:33.024846 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:33.024853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:33.024859 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:33.025269 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:33.025428 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:33.026160 (+ 732us) server_negotiation.cc:299] Negotiation successful
0812 06:56:33.026437 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":265,"threads_started":1}
I20260812 06:56:33.027537 5514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:33.004096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.1:49613)
0812 06:56:33.005131 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:33.005221 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:33.007631 (+ 2410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:33.008474 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:33.008488 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:33.009465 (+ 977us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:33.010426 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:33.010458 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.011544 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.011553 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:33.014208 (+ 2655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:33.014226 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.015128 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.015138 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:33.015487 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:33.016346 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:33.016383 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:33.019390 (+ 3007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:33.023565 (+ 4175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:33.023573 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:33.023601 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:33.024028 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:33.024510 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:33.024513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:33.024515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:33.024724 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:33.025399 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:33.025405 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:33.025746 (+ 341us) client_negotiation.cc:769] Sending connection context
0812 06:56:33.025991 (+ 245us) client_negotiation.cc:240] Negotiation successful
0812 06:56:33.026391 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":422,"threads_started":1}
I20260812 06:56:33.029318 5507 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36667
I20260812 06:56:33.029664 5507 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:33.030668 5507 heartbeater.cc:511] Master 127.26.226.62:36667 requested a full tablet report, sending...
I20260812 06:56:33.033003 5210 ts_manager.cc:194] Registered new tserver with Master: 37919c2b404d4ac4b9d7e5f2e4ec0174 (127.26.226.1:42641)
I20260812 06:56:33.034574 5210 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:49613
WARNING: no policy specified for oryx/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab.
Entry for principal oryx/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab.
I20260812 06:56:33.069473 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36667
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab
--principal=oryx/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:33.411556 5518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:33.411862 5518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:33.412154 5518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:33.419672 5518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:33.419782 5518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:33.419953 5518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:33.429832 5518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab
--principal=oryx/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:33.431232 5518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:33.432629 5518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:33.446019 5534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:33.446110 5528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:33.446064 5526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:33.446918 5518 server_base.cc:1034] running on GCE node
I20260812 06:56:33.447556 5518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:33.448577 5518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:33.449810 5518 hybrid_clock.cc:648] HybridClock initialized: now 1786517793449749 us; error 52 us; skew 500 ppm
I20260812 06:56:33.450121 5518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:33 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517793, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:33.453522 5518 init.cc:377] Logged in from keytab as oryx/127.26.226.2@KRBTEST.COM (short username oryx)
I20260812 06:56:33.454926 5518 webserver.cc:460] Webserver started at http://127.26.226.2:42359/ using document root <none> and password file <none>
I20260812 06:56:33.455613 5518 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:33.455760 5518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:33.456117 5518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:33.459002 5518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "20b470bb7e1a457eba01ba8453cf2a7e"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-t3q3"
server_key: "59265eef7d4c62bccd3b6f3b9b3955d4"
server_key_iv: "2d28b22bf8a9869b79d57896dda294ee"
server_key_version: "encryptionkey@0"
I20260812 06:56:33.459786 5518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "20b470bb7e1a457eba01ba8453cf2a7e"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-t3q3"
server_key: "59265eef7d4c62bccd3b6f3b9b3955d4"
server_key_iv: "2d28b22bf8a9869b79d57896dda294ee"
server_key_version: "encryptionkey@0"
I20260812 06:56:33.464965 5518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 06:56:33.469112 5542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:33.470427 5518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:33.470607 5518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "20b470bb7e1a457eba01ba8453cf2a7e"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-t3q3"
server_key: "59265eef7d4c62bccd3b6f3b9b3955d4"
server_key_iv: "2d28b22bf8a9869b79d57896dda294ee"
server_key_version: "encryptionkey@0"
I20260812 06:56:33.470819 5518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:33.486889 5518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:33.491055 5518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:33.491376 5518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:33.492404 5518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:33.493860 5518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:33.493976 5518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:33.494133 5518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:33.494210 5518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:33.528192 5518 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33865
I20260812 06:56:33.528227 5729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33865 every 8 connection(s)
I20260812 06:56:33.529563 5518 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:33.534924 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 5518
I20260812 06:56:33.535128 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:33.535714 27528 external_mini_cluster.cc:1428] Setting key 730c74c557664896e7114511b1137ffe
Aug 12 06:56:33 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517793, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:33.552171 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:33.532179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:54457 (local address 127.26.226.62:36667)
0812 06:56:33.532624 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:33.532629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:33.533620 (+ 991us) server_negotiation.cc:407] Connection header received
0812 06:56:33.536048 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:33.536053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:33.536168 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:33.536269 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:33.537822 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.538714 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.540727 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.541151 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.541281 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:33.545162 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:33.545171 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:33.545178 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:33.545226 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:33.548870 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:33.549632 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:33.549642 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:33.549644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:33.549779 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:33.550238 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:33.550247 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:33.550251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:33.550535 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:33.550795 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:33.551503 (+ 708us) server_negotiation.cc:299] Negotiation successful
0812 06:56:33.551753 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":243,"threads_started":1}
I20260812 06:56:33.553190 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:33.532664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.2:54457)
0812 06:56:33.533444 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:33.533483 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:33.535734 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:33.536555 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:33.536563 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:33.537082 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:33.537648 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:33.537664 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.538939 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.538948 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:33.540572 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:33.540588 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.541394 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.541402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:33.541631 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:33.542330 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:33.542352 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:33.544921 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:33.549026 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:33.549035 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:33.549053 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:33.549498 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:33.549900 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:33.549905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:33.549909 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:33.550113 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:33.550692 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:33.550701 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:33.551103 (+ 402us) client_negotiation.cc:769] Sending connection context
0812 06:56:33.551391 (+ 288us) client_negotiation.cc:240] Negotiation successful
0812 06:56:33.551974 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":351,"threads_started":1}
I20260812 06:56:33.554716 5730 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36667
I20260812 06:56:33.555042 5730 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:33.555833 5730 heartbeater.cc:511] Master 127.26.226.62:36667 requested a full tablet report, sending...
I20260812 06:56:33.557447 5210 ts_manager.cc:194] Registered new tserver with Master: 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865)
I20260812 06:56:33.558279 5210 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:54457
WARNING: no policy specified for oryx/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab.
Entry for principal oryx/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab.
I20260812 06:56:33.595763 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36667
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab
--principal=oryx/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:33.950948 5741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:33.951316 5741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:33.951488 5741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:33.958842 5741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:33.959010 5741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:33.959183 5741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:33.970369 5741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab
--principal=oryx/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:33.971751 5741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:33.973178 5741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:33.986912 5749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:33.986961 5751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:33.986881 5754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:33.987397 5741 server_base.cc:1034] running on GCE node
I20260812 06:56:33.988214 5741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:33.989158 5741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:33.990376 5741 hybrid_clock.cc:648] HybridClock initialized: now 1786517793990313 us; error 54 us; skew 500 ppm
I20260812 06:56:33.990723 5741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:33 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517793, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:33.994088 5741 init.cc:377] Logged in from keytab as oryx/127.26.226.3@KRBTEST.COM (short username oryx)
I20260812 06:56:33.995615 5741 webserver.cc:460] Webserver started at http://127.26.226.3:44371/ using document root <none> and password file <none>
I20260812 06:56:33.996397 5741 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:33.996533 5741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:33.996860 5741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:33.999893 5741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "c79032ca1b154df28ff4759562daa6e5"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-t3q3"
server_key: "408e1f56c344e05be1af958c19961216"
server_key_iv: "49575fe07e48dd4e112a6af4e8988ffa"
server_key_version: "encryptionkey@0"
I20260812 06:56:34.000818 5741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "c79032ca1b154df28ff4759562daa6e5"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-t3q3"
server_key: "408e1f56c344e05be1af958c19961216"
server_key_iv: "49575fe07e48dd4e112a6af4e8988ffa"
server_key_version: "encryptionkey@0"
I20260812 06:56:34.006071 5741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:56:34.010037 5762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:34.011615 5741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:56:34.011850 5741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "c79032ca1b154df28ff4759562daa6e5"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-t3q3"
server_key: "408e1f56c344e05be1af958c19961216"
server_key_iv: "49575fe07e48dd4e112a6af4e8988ffa"
server_key_version: "encryptionkey@0"
I20260812 06:56:34.012132 5741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:34.032678 5741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:34.037883 5507 heartbeater.cc:503] Master 127.26.226.62:36667 was elected leader, sending a full tablet report...
I20260812 06:56:34.042696 5741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:34.042972 5741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:34.044011 5741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:34.045682 5741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:34.045789 5741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:34.045943 5741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:34.046015 5741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:34.080426 5741 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:46431
I20260812 06:56:34.080467 5939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:46431 every 8 connection(s)
I20260812 06:56:34.081825 5741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:34.082744 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 5741
I20260812 06:56:34.082943 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:34.083604 27528 external_mini_cluster.cc:1428] Setting key 6aa4357ce96eca71cb85bfa633bc383c
Aug 12 06:56:34 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517793, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:34.103048 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.084936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:56857 (local address 127.26.226.62:36667)
0812 06:56:34.085454 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:34.085458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:34.086739 (+ 1281us) server_negotiation.cc:407] Connection header received
0812 06:56:34.088741 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:34.088745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:34.088812 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:34.088898 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:34.090512 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.091329 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.092922 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.093345 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.093444 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.096903 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:34.096908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:34.096911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:34.096945 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:34.100236 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.101007 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.101011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.101013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.101136 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.101528 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.101531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.101533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.101815 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:34.101928 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:34.102593 (+ 665us) server_negotiation.cc:299] Negotiation successful
0812 06:56:34.102770 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":260,"threads_started":1}
I20260812 06:56:34.104202 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.085656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.3:56857)
0812 06:56:34.086559 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:34.086615 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:34.088474 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:34.089109 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:34.089117 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:34.089768 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:34.090348 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.090366 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.091467 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.091471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.092797 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.092810 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.093541 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.093546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.093750 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.094408 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:34.094436 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:34.096701 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:34.100399 (+ 3698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.100406 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.100419 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.100887 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.101232 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.101235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.101238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.101425 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.101958 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:34.101964 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:34.102277 (+ 313us) client_negotiation.cc:769] Sending connection context
0812 06:56:34.102509 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 06:56:34.103042 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":367,"threads_started":1}
I20260812 06:56:34.105533 5940 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36667
I20260812 06:56:34.105798 5940 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:34.106528 5940 heartbeater.cc:511] Master 127.26.226.62:36667 requested a full tablet report, sending...
I20260812 06:56:34.107973 5210 ts_manager.cc:194] Registered new tserver with Master: c79032ca1b154df28ff4759562daa6e5 (127.26.226.3:46431)
I20260812 06:56:34.109074 5210 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:56857
I20260812 06:56:34.112102 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.226.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.61.keytab.
Entry for principal oryx/127.26.226.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.61.keytab.
I20260812 06:56:34.172447 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.61:43543
--webserver_interface=127.26.226.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.61.keytab
--principal=oryx/127.26.226.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:36667,127.26.226.61:43543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:34.526708 5950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:34.527040 5950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:34.527228 5950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:34.536965 5950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:34.537074 5950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:34.537122 5950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:34.537199 5950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:34.537312 5950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:34.547022 5950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:36667,127.26.226.61:43543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.61.keytab
--principal=oryx/127.26.226.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.61:43543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.226.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
W20260812 06:56:34.547328 5950 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.26.226.62:36667,127.26.226.61:43543'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 06:56:34.548424 5950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:34.549662 5950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:56:34.561017 5730 heartbeater.cc:503] Master 127.26.226.62:36667 was elected leader, sending a full tablet report...
W20260812 06:56:34.561510 5957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:34.561859 5960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:34.562047 5956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:34.563154 5950 server_base.cc:1034] running on GCE node
I20260812 06:56:34.563939 5950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:34.565382 5950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:34.566609 5950 hybrid_clock.cc:648] HybridClock initialized: now 1786517794566532 us; error 57 us; skew 500 ppm
I20260812 06:56:34.566939 5950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:34 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517794, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:34.570166 5950 init.cc:377] Logged in from keytab as oryx/127.26.226.61@KRBTEST.COM (short username oryx)
I20260812 06:56:34.571599 5950 webserver.cc:460] Webserver started at http://127.26.226.61:37541/ using document root <none> and password file <none>
I20260812 06:56:34.572383 5950 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:34.572516 5950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:34.572834 5950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:34.576022 5950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data/instance:
uuid: "10df32c12dff4c2e91adacad56bc3efb"
format_stamp: "Formatted at 2026-08-12 06:56:34 on dist-test-slave-t3q3"
server_key: "bda36a7efc71084b081f0b4a46ba5658"
server_key_iv: "79f476c33995efd1c428b11b5305bcef"
server_key_version: "encryptionkey@0"
I20260812 06:56:34.576879 5950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal/instance:
uuid: "10df32c12dff4c2e91adacad56bc3efb"
format_stamp: "Formatted at 2026-08-12 06:56:34 on dist-test-slave-t3q3"
server_key: "bda36a7efc71084b081f0b4a46ba5658"
server_key_iv: "79f476c33995efd1c428b11b5305bcef"
server_key_version: "encryptionkey@0"
I20260812 06:56:34.582361 5950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:56:34.585855 5972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:34.587236 5950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:34.587426 5950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal
uuid: "10df32c12dff4c2e91adacad56bc3efb"
format_stamp: "Formatted at 2026-08-12 06:56:34 on dist-test-slave-t3q3"
server_key: "bda36a7efc71084b081f0b4a46ba5658"
server_key_iv: "79f476c33995efd1c428b11b5305bcef"
server_key_version: "encryptionkey@0"
I20260812 06:56:34.587656 5950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:34.602077 5950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:34.606518 5950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:34.606803 5950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:34.625952 5950 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.61:43543
I20260812 06:56:34.625976 6058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.61:43543 every 8 connection(s)
I20260812 06:56:34.627393 5950 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
I20260812 06:56:34.629379 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 5950
I20260812 06:56:34.632100 6060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:34.640733 6060 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:56:34 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.61@KRBTEST.COM
Aug 12 06:56:34 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517794, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.61@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:34.653877 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.631693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.26.226.61:43543)
0812 06:56:34.632579 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:34.632591 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:34.632632 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:56:34.634513 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:34.634534 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:34.634914 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:34.635246 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:34.637208 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.638560 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.640357 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.641047 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.641179 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.646378 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:34.646389 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:34.646399 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:34.646444 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:34.649417 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.650120 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.650126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.650132 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.650383 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.650796 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.650801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.650804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.651304 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:34.651686 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:34.652154 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 06:56:34.652555 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":468,"threads_started":1}
I20260812 06:56:34.658767 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.643733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:38694)
0812 06:56:34.644531 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:34.644584 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:34.644784 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:34.645438 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:34.645445 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:34.646190 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:34.646529 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.646540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.647767 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.647773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.648982 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.648995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.649760 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.649766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.649935 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.650532 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:34.650556 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:34.652792 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:34.656248 (+ 3456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.656257 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.656263 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.656899 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.657293 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.657297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.657299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.657418 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.657975 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:34.657980 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:34.658121 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 06:56:34.658269 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 06:56:34.658489 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":417,"threads_started":1}
I20260812 06:56:34.658780 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.643831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.26.226.62:36667)
0812 06:56:34.644997 (+ 1166us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:34.645006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:34.645029 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:56:34.645097 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:34.645105 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:34.645211 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:34.645332 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:34.646696 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.647617 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.649144 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.649571 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.649724 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.653000 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:34.653008 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:34.653013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:34.653059 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:34.656126 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.657046 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.657051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.657054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.657187 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.657526 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.657529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.657531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.657846 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:34.657970 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:34.658367 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 06:56:34.658532 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":1039,"threads_started":1}
I20260812 06:56:34.660377 6060 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:56:34 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517794, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.61@KRBTEST.COM for oryx/127.26.226.61@KRBTEST.COM
I20260812 06:56:34.671960 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.661001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:43543 (local address 127.0.0.1:33154)
0812 06:56:34.661177 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:34.661192 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:34.661266 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:34.661540 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:34.661545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:34.661847 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:34.661972 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.661979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.662945 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.662949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.663807 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.663818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.664546 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.664550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.664700 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.665198 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:34.665214 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:34.667170 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:34.670009 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.670013 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.670015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.670379 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.670701 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.670704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.670706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.670828 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.671276 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:34.671279 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:34.671406 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 06:56:34.671517 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 06:56:34.671720 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 06:56:34.672226 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.661180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33154 (local address 127.26.226.61:43543)
0812 06:56:34.661320 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:34.661324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:34.661338 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:56:34.661375 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:34.661378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:34.661427 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:34.661502 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:34.662103 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.662789 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.663930 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.664391 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.664487 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.667333 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:34.667337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:34.667340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:34.667368 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:34.669885 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.670483 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.670487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.670489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.670607 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.670915 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.670918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.670920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.671167 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:34.671287 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:34.671603 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 06:56:34.671855 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":52}
I20260812 06:56:34.677206 6060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: Bootstrap starting.
I20260812 06:56:34.680382 6060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:34.681401 6060 log.cc:824] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:34.684046 6060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: No bootstrap required, opened a new log
I20260812 06:56:34.691131 6060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } }
I20260812 06:56:34.691506 6060 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:34.691643 6060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10df32c12dff4c2e91adacad56bc3efb, State: Initialized, Role: FOLLOWER
I20260812 06:56:34.692317 6060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } }
I20260812 06:56:34.693243 6068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } } }
I20260812 06:56:34.693588 6068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:56:34.694290 6060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:56:34.703624 6088 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:56:34.703894 6088 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:56:34.706174 5950 master_runner.cc:428] Detected that this master 10df32c12dff4c2e91adacad56bc3efb is joining an existing cluster
I20260812 06:56:34.706277 5950 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.226.61:43543
I20260812 06:56:34.737773 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.711236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:38706)
0812 06:56:34.720634 (+ 9398us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:34.720659 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:34.720945 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:34.721622 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:34.721627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:34.722389 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:34.722828 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.722842 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.724448 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.724456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.726101 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.726115 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.727413 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.727421 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.727623 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.728459 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:34.728484 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:34.729756 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:34.734241 (+ 4485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.734246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.734250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.734862 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.735345 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.735350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.735354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.735552 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.736422 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:34.736428 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:34.736659 (+ 231us) client_negotiation.cc:769] Sending connection context
0812 06:56:34.736872 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 06:56:34.737227 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9215,"thread_start_us":9036,"threads_started":1}
I20260812 06:56:34.739197 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.711280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38706 (local address 127.26.226.62:36667)
0812 06:56:34.711470 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:34.711477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:34.720794 (+ 9317us) server_negotiation.cc:407] Connection header received
0812 06:56:34.721130 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:34.721139 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:34.721249 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:34.721354 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:34.723032 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.724256 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.726269 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.726978 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.727116 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.730003 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:34.730009 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:34.730014 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:34.730058 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:34.734064 (+ 4006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.735023 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.735029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.735033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.735198 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.735723 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.735730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.735736 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.736174 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:34.736348 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:34.738295 (+ 1947us) server_negotiation.cc:299] Negotiation successful
0812 06:56:34.738540 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
WARNING: no policy specified for oryx/127.26.226.60@KRBTEST.COM; defaulting to no policy
I20260812 06:56:34.750491 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.721034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33156 (local address 127.26.226.61:43543)
0812 06:56:34.721257 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:34.721265 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:34.723516 (+ 2251us) server_negotiation.cc:407] Connection header received
0812 06:56:34.723734 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:34.723739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:34.723810 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:34.723877 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:34.725405 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.726398 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.736463 (+ 10065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.737290 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.737409 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.743111 (+ 5702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:34.743119 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:34.743126 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:34.743165 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:34.747166 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.748153 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.748159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.748162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.748309 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.748769 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.748774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.748778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.749178 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:34.749342 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:34.749880 (+ 538us) server_negotiation.cc:299] Negotiation successful
0812 06:56:34.750112 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":65}
I20260812 06:56:34.751639 6097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.721389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:43543 (local address 127.0.0.1:33156)
0812 06:56:34.723425 (+ 2036us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:34.723444 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:34.723603 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:34.724104 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:34.724109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:34.724865 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:34.725224 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.725238 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.726581 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.726588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.736297 (+ 9709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.736316 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.738174 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.738189 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.738438 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.740226 (+ 1788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:34.740255 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:34.741538 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:34.747345 (+ 5807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.747351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.747355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.747983 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.748460 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.748464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.748468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.748642 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.749417 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:34.749422 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:34.749572 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 06:56:34.750858 (+ 1286us) client_negotiation.cc:240] Negotiation successful
0812 06:56:34.751183 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1904,"thread_start_us":1708,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.60@KRBTEST.COM" created.
Aug 12 06:56:34 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517792, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.62@KRBTEST.COM for oryx/127.26.226.61@KRBTEST.COM
I20260812 06:56:34.772776 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.747987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33166 (local address 127.26.226.61:43543)
0812 06:56:34.750820 (+ 2833us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:34.750826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:34.754131 (+ 3305us) server_negotiation.cc:407] Connection header received
0812 06:56:34.754471 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:34.754477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:34.754613 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:34.754732 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:34.757353 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.758541 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.760623 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.761311 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.761466 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.765483 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:34.765489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:34.765495 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:34.765533 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:34.769300 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.770323 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.770333 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.770337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.770498 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.770968 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.770974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.770977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.771329 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:34.771477 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:34.772141 (+ 664us) server_negotiation.cc:299] Negotiation successful
0812 06:56:34.772367 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2723,"thread_start_us":5342,"threads_started":1}
I20260812 06:56:34.772926 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.747506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:43543 (local address 127.0.0.1:33166)
0812 06:56:34.753971 (+ 6465us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:34.754020 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:34.754225 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:34.754817 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:34.754829 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:34.756829 (+ 2000us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:34.757168 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.757184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.758695 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.758701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.760454 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.760473 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.761520 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.761529 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.761771 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.762571 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:34.762605 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:34.765260 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:34.769458 (+ 4198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.769473 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.769483 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.770169 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.770628 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.770635 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.770641 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.770828 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.771485 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:34.771497 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:34.771734 (+ 237us) client_negotiation.cc:769] Sending connection context
0812 06:56:34.771972 (+ 238us) client_negotiation.cc:240] Negotiation successful
0812 06:56:34.772384 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6180,"thread_start_us":8618,"threads_started":1}
I20260812 06:56:34.775429 5210 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.226.61:43543: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb"
member_type: NON_VOTER
last_known_addr {
host: "127.26.226.61"
port: 43543
}
attrs {
promote: true
}
}
dest_uuid: "f4a6c926fc984692809b846da0948692"
cas_config_opid_index: -1
I20260812 06:56:34.777240 5210 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } }
I20260812 06:56:34.781909 6108 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 10df32c12dff4c2e91adacad56bc3efb (127.26.226.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } } }
I20260812 06:56:34.783398 6015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:34.787685 6015 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 FOLLOWER]: Refusing update from remote peer f4a6c926fc984692809b846da0948692: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 06:56:34.786731 6108 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.226.61:43543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.60.keytab.
Entry for principal oryx/127.26.226.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.60.keytab.
I20260812 06:56:34.789011 5950 master.cc:537] Master@127.26.226.61:43543 shutting down...
I20260812 06:56:34.790333 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.60:33887
--webserver_interface=127.26.226.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.60.keytab
--principal=oryx/127.26.226.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:36667,127.26.226.61:43543,127.26.226.60:33887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:56:34.798213 6108 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:34.801738 5950 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:34.802242 5950 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:34.802520 5950 tablet_replica.cc:330] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: stopping tablet replica
I20260812 06:56:34.803165 6110 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } } }
I20260812 06:56:34.803542 6110 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: This master's current role is: LEADER
W20260812 06:56:34.804944 5182 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 06:56:34.807034 5182 consensus_peers.cc:487] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 -> Peer 10df32c12dff4c2e91adacad56bc3efb (127.26.226.61:43543): Couldn't send request to peer 10df32c12dff4c2e91adacad56bc3efb. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260812 06:56:34.820729 5950 master.cc:552] Master@127.26.226.61:43543 shutdown complete.
I20260812 06:56:34.820909 5950 master_runner.cc:305] Clearing existing system tablet
I20260812 06:56:34.821422 5950 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:34.821532 5950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:34.824927 5950 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.003s
I20260812 06:56:34.826779 6121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:34.827169 5950 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:34.827445 5950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 06:56:34.827535 5950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal
uuid: "10df32c12dff4c2e91adacad56bc3efb"
format_stamp: "Formatted at 2026-08-12 06:56:34 on dist-test-slave-t3q3"
server_key: "bda36a7efc71084b081f0b4a46ba5658"
server_key_iv: "79f476c33995efd1c428b11b5305bcef"
server_key_version: "encryptionkey@0"
I20260812 06:56:34.833103 5950 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:34.840478 5950 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 06:56:34.840679 5950 log.cc:1196] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 06:56:34.841125 5950 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: Deleting consensus metadata
I20260812 06:56:34.841395 5950 master_runner.cc:315] Copying system tablet from 127.26.226.62:36667
I20260812 06:56:34.844817 5950 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: tablet copy: Beginning tablet copy session from remote peer at address 127.26.226.62:36667
I20260812 06:56:34.861933 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.845820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.26.226.62:36667)
0812 06:56:34.846027 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:34.846037 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:34.846486 (+ 449us) server_negotiation.cc:407] Connection header received
0812 06:56:34.846715 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:34.846721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:34.846805 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:34.846910 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:34.848321 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.849476 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.851424 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.852294 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.852413 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.854997 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:34.855005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:34.855012 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:34.855051 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:34.858704 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.859555 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.859562 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.859566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.859720 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:34.860222 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:34.860229 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:34.860234 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:34.860652 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:34.860818 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:34.861318 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 06:56:34.861563 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:56:34.861987 6130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:34.845783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:38712)
0812 06:56:34.846350 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:34.846378 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:34.846540 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:34.847001 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:34.847006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:34.847775 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:34.848132 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.848146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.849677 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.849682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.851213 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:34.851233 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:34.852722 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:34.852728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:34.852920 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:34.853650 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:34.853675 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:34.854735 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:34.858871 (+ 4136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.858878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.858882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.859415 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.859846 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:34.859857 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:34.859861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:34.860077 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:34.860785 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:34.860790 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:34.860988 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 06:56:34.861180 (+ 192us) client_negotiation.cc:240] Negotiation successful
0812 06:56:34.861506 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":318,"threads_started":1}
I20260812 06:56:34.863348 5245 tablet_copy_service.cc:140] P f4a6c926fc984692809b846da0948692: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 10df32c12dff4c2e91adacad56bc3efb ({username='oryx', principal='oryx/127.26.226.61@KRBTEST.COM'} at 127.0.0.1:38712)
I20260812 06:56:34.863775 5245 tablet_copy_service.cc:161] P f4a6c926fc984692809b846da0948692: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 10df32c12dff4c2e91adacad56bc3efb at {username='oryx', principal='oryx/127.26.226.61@KRBTEST.COM'} at 127.0.0.1:38712: session id = 10df32c12dff4c2e91adacad56bc3efb-00000000000000000000000000000000
I20260812 06:56:34.869663 5245 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 06:56:34.872786 5950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 06:56:34.880167 5950 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: tablet copy: Starting download of 0 data blocks...
I20260812 06:56:34.880579 5950 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: tablet copy: Starting download of 1 WAL segments...
I20260812 06:56:34.884235 5950 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 06:56:34.888963 5245 tablet_copy_service.cc:342] P f4a6c926fc984692809b846da0948692: Request end of tablet copy session 10df32c12dff4c2e91adacad56bc3efb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.226.61@KRBTEST.COM'} at 127.0.0.1:38712
I20260812 06:56:34.889259 5245 tablet_copy_service.cc:433] P f4a6c926fc984692809b846da0948692: ending tablet copy session 10df32c12dff4c2e91adacad56bc3efb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 10df32c12dff4c2e91adacad56bc3efb
I20260812 06:56:34.902622 5950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:34.905334 6131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:34.909420 6134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:34.918768 5950 server_base.cc:1034] running on GCE node
W20260812 06:56:34.919523 6139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:34.920025 5950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:34.920805 5950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:34.922212 5950 hybrid_clock.cc:648] HybridClock initialized: now 1786517794922147 us; error 42 us; skew 500 ppm
I20260812 06:56:34.922451 5950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:34 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517794, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:34.924700 5950 init.cc:377] Logged in from keytab as oryx/127.26.226.61@KRBTEST.COM (short username oryx)
I20260812 06:56:34.929039 5950 webserver.cc:460] Webserver started at http://127.26.226.61:41491/ using document root <none> and password file <none>
I20260812 06:56:34.929418 5950 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:34.929505 5950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:34.932778 5950 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 06:56:34.934950 6149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:34.935420 5950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 06:56:34.935565 5950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal
uuid: "10df32c12dff4c2e91adacad56bc3efb"
format_stamp: "Formatted at 2026-08-12 06:56:34 on dist-test-slave-t3q3"
server_key: "bda36a7efc71084b081f0b4a46ba5658"
server_key_iv: "79f476c33995efd1c428b11b5305bcef"
server_key_version: "encryptionkey@0"
I20260812 06:56:34.935745 5950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:34.974375 5950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:34.978722 5950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:35.008680 5950 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.61:43543
I20260812 06:56:35.008692 6240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.61:43543 every 8 connection(s)
I20260812 06:56:35.009686 5950 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
I20260812 06:56:35.016885 6243 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:56:35.020988 6243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: Bootstrap starting.
I20260812 06:56:35.035619 6243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:56:35.036314 6243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: Bootstrap complete.
I20260812 06:56:35.037370 6243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } }
I20260812 06:56:35.037642 6243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10df32c12dff4c2e91adacad56bc3efb, State: Initialized, Role: LEARNER
I20260812 06:56:35.037909 6243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } }
I20260812 06:56:35.038748 6246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } } }
I20260812 06:56:35.039075 6243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:35.039121 6246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: This master's current role is: LEARNER
I20260812 06:56:35.051895 6263 catalog_manager.cc:1259] Loaded cluster ID: 993356870845481c94b7918d6e35d6eb
I20260812 06:56:35.052039 6263 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: loading cluster ID for follower catalog manager: success
I20260812 06:56:35.055755 6263 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: acquiring CA information for follower catalog manager: success
I20260812 06:56:35.058429 6263 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:56:35.112600 5940 heartbeater.cc:503] Master 127.26.226.62:36667 was elected leader, sending a full tablet report...
W20260812 06:56:35.199507 6117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:35.199792 6117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:35.199946 6117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:35.207332 6117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:35.207448 6117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:35.207553 6117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:35.207651 6117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:35.207726 6117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:35.218288 6117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:36667,127.26.226.61:43543,127.26.226.60:33887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.60.keytab
--principal=oryx/127.26.226.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.60:33887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.226.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:35.219791 6117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:35.221187 6117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:35.233227 6276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:35.233515 6280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:35.233594 6277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:35.234689 6117 server_base.cc:1034] running on GCE node
I20260812 06:56:35.235538 6117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:35.237076 6117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:35.238281 6117 hybrid_clock.cc:648] HybridClock initialized: now 1786517795238222 us; error 82 us; skew 500 ppm
I20260812 06:56:35.238646 6117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:35 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517795, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:35.246601 6117 init.cc:377] Logged in from keytab as oryx/127.26.226.60@KRBTEST.COM (short username oryx)
I20260812 06:56:35.248298 6117 webserver.cc:460] Webserver started at http://127.26.226.60:46471/ using document root <none> and password file <none>
I20260812 06:56:35.249044 6117 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:35.249191 6117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:35.249500 6117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:35.252725 6117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data/instance:
uuid: "e2e89365619e4408b2ad53aee78acff8"
format_stamp: "Formatted at 2026-08-12 06:56:35 on dist-test-slave-t3q3"
server_key: "bc20a016fd25c6e576531d0d97db02e1"
server_key_iv: "cdb73f2faed6d70bfe572276c17d3504"
server_key_version: "encryptionkey@0"
I20260812 06:56:35.253496 6117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal/instance:
uuid: "e2e89365619e4408b2ad53aee78acff8"
format_stamp: "Formatted at 2026-08-12 06:56:35 on dist-test-slave-t3q3"
server_key: "bc20a016fd25c6e576531d0d97db02e1"
server_key_iv: "cdb73f2faed6d70bfe572276c17d3504"
server_key_version: "encryptionkey@0"
I20260812 06:56:35.258937 6117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260812 06:56:35.262537 6288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:35.263893 6117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:56:35.264145 6117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal
uuid: "e2e89365619e4408b2ad53aee78acff8"
format_stamp: "Formatted at 2026-08-12 06:56:35 on dist-test-slave-t3q3"
server_key: "bc20a016fd25c6e576531d0d97db02e1"
server_key_iv: "cdb73f2faed6d70bfe572276c17d3504"
server_key_version: "encryptionkey@0"
I20260812 06:56:35.264349 6117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:35.292449 6117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:35.296527 6117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:35.296823 6117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:35.316656 6117 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.60:33887
I20260812 06:56:35.316725 6370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.60:33887 every 8 connection(s)
I20260812 06:56:35.318216 6117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
I20260812 06:56:35.321338 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 6117
I20260812 06:56:35.322574 6372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:35.332589 6372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:56:35 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.60@KRBTEST.COM
Aug 12 06:56:35 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517795, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.60@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:35.348366 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.323662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.26.226.60:33887)
0812 06:56:35.324470 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.324482 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.324524 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:56:35.326072 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.326094 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.326472 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.326824 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.328770 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.330190 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.331978 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.332723 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.332878 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.338850 (+ 5972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.338858 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.338868 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:35.338914 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.342505 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.343405 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.343411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.343418 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.343591 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.344141 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.344146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.344150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.344804 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.345040 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.346393 (+ 1353us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.346899 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":384,"threads_started":1}
I20260812 06:56:35.353709 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.335959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.26.226.62:36667)
0812 06:56:35.336235 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.336244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.337041 (+ 797us) server_negotiation.cc:407] Connection header received
0812 06:56:35.337303 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.337308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.337387 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.337494 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.339177 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.340287 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.342320 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.342966 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.343108 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.347513 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.347518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.347521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:35.347556 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.350618 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.351484 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.351492 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.351498 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.351629 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.352004 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.352010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.352013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.352427 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.352619 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.353130 (+ 511us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.353361 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260812 06:56:35.353793 6380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.335898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:38722)
0812 06:56:35.336878 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:35.336933 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:35.337132 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:35.337596 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:35.337603 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:35.338605 (+ 1002us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:35.339007 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.339019 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.340495 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.340500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.342147 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.342172 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.343198 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.343206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.343439 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.344254 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:35.344273 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:35.347320 (+ 3047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:35.350763 (+ 3443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.350772 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.350785 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.351351 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.351744 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.351749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.351752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.351889 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.352598 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:35.352605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:35.352785 (+ 180us) client_negotiation.cc:769] Sending connection context
0812 06:56:35.353002 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 06:56:35.353354 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"spinlock_wait_cycles":16640,"thread_start_us":549,"threads_started":1}
I20260812 06:56:35.355072 6372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:56:35 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517795, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.60@KRBTEST.COM for oryx/127.26.226.61@KRBTEST.COM
I20260812 06:56:35.370730 6380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.355696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:43543 (local address 127.0.0.1:33180)
0812 06:56:35.355915 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:35.355935 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:35.356036 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:35.356760 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:35.356764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:35.357068 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:35.357205 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.357215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.358517 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.358520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.359430 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.359440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.360268 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.360272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.360412 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.360919 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:35.360935 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:35.362894 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:35.367018 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.367024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.367028 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.367550 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.368239 (+ 689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.368246 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.368320 (+ 74us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.368629 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.369608 (+ 979us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:35.369615 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:35.369812 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 06:56:35.369996 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 06:56:35.370356 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260812 06:56:35.371755 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.355915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33180 (local address 127.26.226.61:43543)
0812 06:56:35.356487 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.356492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.356508 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:56:35.356557 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.356561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.356618 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.356714 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.357352 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.358396 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.359602 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.360040 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.360244 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.363099 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.363105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.363108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:35.363144 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.366851 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.367697 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.367704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.367708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.368023 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.368855 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.368862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.368867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.369335 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.369493 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.371084 (+ 1591us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.371338 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":314,"threads_started":1}
I20260812 06:56:35.373034 6372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:56:35 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517795, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.60@KRBTEST.COM for oryx/127.26.226.60@KRBTEST.COM
I20260812 06:56:35.389261 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.374166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.26.226.60:33887)
0812 06:56:35.374325 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.374330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.374348 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:56:35.374511 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.374517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.374583 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.374664 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.375600 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.376488 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.377827 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.378468 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.378611 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.382757 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.382763 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.382767 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:35.382797 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.386081 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.387126 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.387130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.387133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.387252 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.387774 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.387778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.387780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.388044 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.388234 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.388598 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.388832 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:56:35.389374 6380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.373942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:33887 (local address 127.0.0.1:36644)
0812 06:56:35.374154 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:35.374179 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:35.374361 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:35.374792 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:35.374795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:35.375246 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:35.375438 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.375449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.376609 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.376615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.377687 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.377700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.378632 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.378638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.378865 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.379577 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:35.379608 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:35.382582 (+ 2974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:35.386232 (+ 3650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.386239 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.386243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.386983 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.387369 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.387377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.387384 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.387650 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.388209 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:35.388214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:35.388347 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 06:56:35.388506 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 06:56:35.388965 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":129}
I20260812 06:56:35.395615 6372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Bootstrap starting.
I20260812 06:56:35.401122 6372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:35.402653 6372 log.cc:824] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:35.406431 6372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: No bootstrap required, opened a new log
I20260812 06:56:35.419950 6372 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } }
I20260812 06:56:35.420382 6372 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:35.420533 6372 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e89365619e4408b2ad53aee78acff8, State: Initialized, Role: FOLLOWER
I20260812 06:56:35.421139 6372 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } }
I20260812 06:56:35.422070 6387 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } } }
I20260812 06:56:35.422482 6387 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:56:35.423134 6372 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:35.427021 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.415574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:43543 (local address 127.0.0.1:33194)
0812 06:56:35.420628 (+ 5054us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:35.420666 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:35.420824 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:35.421256 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:35.421260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:35.421288 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:35.421482 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.421493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.422990 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.422993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.424542 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.424556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.426088 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.426093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.426276 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.426336 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 06:56:35.426480 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 06:56:35.426651 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4933,"thread_start_us":4692,"threads_started":1}
I20260812 06:56:35.427599 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.415647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33194 (local address 127.26.226.61:43543)
0812 06:56:35.415884 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.415894 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.420758 (+ 4864us) server_negotiation.cc:407] Connection header received
0812 06:56:35.420974 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.420979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.421084 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.421197 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:35.421690 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.422823 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.424697 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.425885 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.426044 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.426250 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.427242 (+ 992us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.427402 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:56:35.428872 6197 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260812 06:56:35.429586 6246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4a6c926fc984692809b846da0948692. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } } }
I20260812 06:56:35.429837 6246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: This master's current role is: LEARNER
W20260812 06:56:35.438184 6404 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:56:35.438334 6404 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:56:35.442102 6117 master_runner.cc:428] Detected that this master e2e89365619e4408b2ad53aee78acff8 is joining an existing cluster
I20260812 06:56:35.442245 6117 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.226.60:33887
I20260812 06:56:35.454875 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 5303
I20260812 06:56:35.470510 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.450630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38732 (local address 127.26.226.62:36667)
0812 06:56:35.450855 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.450864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.451716 (+ 852us) server_negotiation.cc:407] Connection header received
0812 06:56:35.452015 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.452023 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.452149 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.452235 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.453980 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.455085 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.457389 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.458316 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.458452 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.462006 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.462017 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.462025 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:35.462085 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.466388 (+ 4303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.467421 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.467426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.467430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.467611 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.468257 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.468265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.468270 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.468678 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.468826 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.469710 (+ 884us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.470012 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 06:56:35.472141 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.450640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:38732)
0812 06:56:35.451598 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:35.451620 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:35.451833 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:35.452519 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:35.452527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:35.453375 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:35.453789 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.453803 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.455319 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.455324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.457175 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.457197 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.458826 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.458835 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.459109 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.460072 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:35.460103 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:35.461717 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:35.466610 (+ 4893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.466618 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.466624 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.467261 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.467794 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.467802 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.467808 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.468090 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.468946 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:35.468957 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:35.469246 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 06:56:35.470943 (+ 1697us) client_negotiation.cc:240] Negotiation successful
0812 06:56:35.471431 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":485,"threads_started":1}
I20260812 06:56:35.480602 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.450821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.26.226.60:33887)
0812 06:56:35.451009 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.451017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.464795 (+ 13778us) server_negotiation.cc:407] Connection header received
0812 06:56:35.464925 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.464930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.464995 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.465085 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.466501 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.467614 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.469710 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.470415 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.470512 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.473086 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.473095 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.473100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:35.473140 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.477114 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.478124 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.478134 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.478137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.478294 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.478714 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.478719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.478722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.479060 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.479203 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.479727 (+ 524us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.479938 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 06:56:35 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.1@KRBTEST.COM
Aug 12 06:56:35 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517792, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.62@KRBTEST.COM for oryx/127.26.226.60@KRBTEST.COM
I20260812 06:56:35.493299 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.451261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:33887 (local address 127.0.0.1:36654)
0812 06:56:35.464625 (+ 13364us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:35.464640 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:35.464793 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:35.465223 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:35.465229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:35.465975 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:35.466350 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.466364 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.467823 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.467829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.469561 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.469576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.470817 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.470823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.470977 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.471662 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:35.471685 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:35.472882 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:35.477292 (+ 4410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.477299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.477308 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.478002 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.478423 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.478429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.478433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.478578 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.479266 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:35.479270 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:35.479429 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 06:56:35.480304 (+ 875us) client_negotiation.cc:240] Negotiation successful
0812 06:56:35.492780 (+ 12476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13228,"thread_start_us":13028,"threads_started":1}
I20260812 06:56:35.494306 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.451816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.26.226.61:43543)
0812 06:56:35.451998 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.452003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.473020 (+ 21017us) server_negotiation.cc:407] Connection header received
0812 06:56:35.473897 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.473902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.473987 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.474068 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.475953 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.477087 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.480947 (+ 3860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.481587 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.481744 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.484690 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.484696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.484702 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:35.484743 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.488491 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.489352 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.489358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.489363 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.489534 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.490088 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.490092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.490096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.490450 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.490580 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.493656 (+ 3076us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.493930 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:56:35.496390 6419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.452141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:43543 (local address 127.0.0.1:33206)
0812 06:56:35.472849 (+ 20708us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:35.472877 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:35.473604 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:35.474383 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:35.474388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:35.475228 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:35.475753 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.475768 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.477269 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.477273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.480775 (+ 3502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.480789 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.482353 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.482361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.482544 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.483229 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:35.483256 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:35.484464 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:35.488673 (+ 4209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.488677 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.488695 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.489186 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.489704 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.489710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.489716 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.489935 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.490679 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:35.490685 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:35.492534 (+ 1849us) client_negotiation.cc:769] Sending connection context
0812 06:56:35.495406 (+ 2872us) client_negotiation.cc:240] Negotiation successful
0812 06:56:35.495777 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20532,"mutex_wait_us":32,"thread_start_us":20353,"threads_started":1}
I20260812 06:56:35.498181 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.456635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.26.226.1:42641)
0812 06:56:35.473560 (+ 16925us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.473570 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.473601 (+ 31us) server_negotiation.cc:407] Connection header received
0812 06:56:35.473671 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.473682 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.473995 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.476040 (+ 2045us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.476332 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.477811 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.479724 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.481078 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.481213 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.487297 (+ 6084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.487315 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.487336 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:35.487386 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.491443 (+ 4057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.492309 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.492319 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.492322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.492490 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.493078 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.493083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.493087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.493961 (+ 874us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.497248 (+ 3287us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.497442 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.497727 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16748,"thread_start_us":16610,"threads_started":1}
I20260812 06:56:35.499627 5497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:35.501874 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.483930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:33887 (local address 127.0.0.1:36658)
0812 06:56:35.484292 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:35.484312 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:35.484431 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:35.485241 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:35.485246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:35.485705 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:35.485888 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.485900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.487584 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.487590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.489071 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.489088 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.490180 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.490186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.490366 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.490932 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:35.490955 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:35.493665 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:35.498749 (+ 5084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.498755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.498758 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.499292 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.499800 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.499806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.499810 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.499993 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.500833 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:35.500841 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:35.501038 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 06:56:35.501185 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 06:56:35.501499 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244}
I20260812 06:56:35.503230 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.484628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36658 (local address 127.26.226.60:33887)
0812 06:56:35.484840 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.484846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.484862 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:56:35.484946 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.484950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.485056 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.486059 (+ 1003us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.486251 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.487399 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.489273 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.489987 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.490676 (+ 689us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.494634 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.494641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.494645 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:35.494681 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.498602 (+ 3921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.499453 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.499459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.499462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.499613 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.500173 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.500179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.500182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.500620 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.502480 (+ 1860us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.502625 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.502874 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":107}
I20260812 06:56:35.505452 5210 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.226.60:33887: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e2e89365619e4408b2ad53aee78acff8"
member_type: NON_VOTER
last_known_addr {
host: "127.26.226.60"
port: 33887
}
attrs {
promote: true
}
}
dest_uuid: "f4a6c926fc984692809b846da0948692"
cas_config_opid_index: 5
I20260812 06:56:35.506528 5210 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } }
I20260812 06:56:35.508698 6385 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e2e89365619e4408b2ad53aee78acff8 (127.26.226.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:35.509330 6197 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 LEARNER]: Refusing update from remote peer f4a6c926fc984692809b846da0948692: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 06:56:35.513242 6326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:35.517710 6326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 FOLLOWER]: Refusing update from remote peer f4a6c926fc984692809b846da0948692: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 06:56:35.523572 6385 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.226.60:33887
I20260812 06:56:35.523258 6425 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 06:56:35.524287 6117 master.cc:537] Master@127.26.226.60:33887 shutting down...
I20260812 06:56:35.524296 6385 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:35.525044 6426 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:35.527510 6426 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:35.527487 6197 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e2e89365619e4408b2ad53aee78acff8 (127.26.226.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:35.532455 6246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: NON_VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: true } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:35.532763 6246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: This master's current role is: LEARNER
I20260812 06:56:35.534256 6425 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692: attempting to promote NON_VOTER 10df32c12dff4c2e91adacad56bc3efb to VOTER
I20260812 06:56:35.538026 6425 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } }
I20260812 06:56:35.540132 6197 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 LEARNER]: Refusing update from remote peer f4a6c926fc984692809b846da0948692: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 06:56:35.544335 6425 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 06:56:35.547988 6197 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 10df32c12dff4c2e91adacad56bc3efb (127.26.226.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:35.547254 6426 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 10df32c12dff4c2e91adacad56bc3efb (127.26.226.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:35.552342 6246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:35.552605 6246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:56:35.556133 6385 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:35.556416 6385 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: This master's current role is: LEADER
W20260812 06:56:35.556401 6298 tablet.cc:2369] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:35.556902 6385 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:35.557132 6385 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:35.560233 6117 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:35.560644 6117 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:35.560861 6117 tablet_replica.cc:330] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: stopping tablet replica
W20260812 06:56:35.562130 5183 consensus_peers.cc:487] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 -> Peer e2e89365619e4408b2ad53aee78acff8 (127.26.226.60:33887): Couldn't send request to peer e2e89365619e4408b2ad53aee78acff8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 06:56:35.568605 6117 master.cc:552] Master@127.26.226.60:33887 shutdown complete.
I20260812 06:56:35.568698 6117 master_runner.cc:305] Clearing existing system tablet
I20260812 06:56:35.569087 6117 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:35.569221 6117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:35.572249 6117 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:56:35.573956 6433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:35.574378 6117 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:35.574858 6117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 06:56:35.575027 6117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal
uuid: "e2e89365619e4408b2ad53aee78acff8"
format_stamp: "Formatted at 2026-08-12 06:56:35 on dist-test-slave-t3q3"
server_key: "bc20a016fd25c6e576531d0d97db02e1"
server_key_iv: "cdb73f2faed6d70bfe572276c17d3504"
server_key_version: "encryptionkey@0"
I20260812 06:56:35.580279 6117 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:35.587848 6117 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 06:56:35.588213 6117 log.cc:1196] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 06:56:35.588786 6117 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Deleting consensus metadata
I20260812 06:56:35.589119 6117 master_runner.cc:315] Copying system tablet from 127.26.226.62:36667
I20260812 06:56:35.591993 6117 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: tablet copy: Beginning tablet copy session from remote peer at address 127.26.226.62:36667
I20260812 06:56:35.612355 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.593036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38744 (local address 127.26.226.62:36667)
0812 06:56:35.593265 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.593269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.596672 (+ 3403us) server_negotiation.cc:407] Connection header received
0812 06:56:35.596868 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.596874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.596957 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.597071 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.598220 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.599302 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.601190 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.601821 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.601953 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.604511 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.604518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.604523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:35.604570 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.609298 (+ 4728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.610114 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.610120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.610124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.610279 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:35.610783 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:35.610788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:35.610791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:35.611149 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.611296 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.611776 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.612008 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:56:35.612819 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.592938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:38744)
0812 06:56:35.596519 (+ 3581us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:35.596546 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:35.596731 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:35.597122 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:35.597127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:35.597733 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:35.597995 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.598008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.599475 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.599481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.601019 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:35.601037 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.601992 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.601999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:35.602190 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.603025 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:35.603049 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:35.604276 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:35.609440 (+ 5164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.609447 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.609452 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.609960 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.610447 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:35.610452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:35.610456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:35.610645 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:35.611275 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:35.611281 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:35.611480 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 06:56:35.611634 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 06:56:35.611988 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3438,"thread_start_us":3289,"threads_started":1}
I20260812 06:56:35.613422 5245 tablet_copy_service.cc:140] P f4a6c926fc984692809b846da0948692: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e2e89365619e4408b2ad53aee78acff8 ({username='oryx', principal='oryx/127.26.226.60@KRBTEST.COM'} at 127.0.0.1:38744)
I20260812 06:56:35.613611 5245 tablet_copy_service.cc:161] P f4a6c926fc984692809b846da0948692: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e2e89365619e4408b2ad53aee78acff8 at {username='oryx', principal='oryx/127.26.226.60@KRBTEST.COM'} at 127.0.0.1:38744: session id = e2e89365619e4408b2ad53aee78acff8-00000000000000000000000000000000
I20260812 06:56:35.617470 5245 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 06:56:35.619468 6117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 06:56:35.626009 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 5303
I20260812 06:56:35.636909 6117 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: tablet copy: Starting download of 0 data blocks...
I20260812 06:56:35.637290 6117 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: tablet copy: Starting download of 1 WAL segments...
I20260812 06:56:35.640127 6117 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 06:56:35.645370 5245 tablet_copy_service.cc:342] P f4a6c926fc984692809b846da0948692: Request end of tablet copy session e2e89365619e4408b2ad53aee78acff8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.226.60@KRBTEST.COM'} at 127.0.0.1:38744
I20260812 06:56:35.645526 5245 tablet_copy_service.cc:433] P f4a6c926fc984692809b846da0948692: ending tablet copy session e2e89365619e4408b2ad53aee78acff8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e2e89365619e4408b2ad53aee78acff8
I20260812 06:56:35.657624 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:42641
--local_ip_for_outbound_sockets=127.26.226.1
--tserver_master_addrs=127.26.226.62:36667
--webserver_port=44991
--webserver_interface=127.26.226.1
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab
--principal=oryx/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.226.62:36667,127.26.226.61:43543
--tserver_master_addrs=127.26.226.62:36667,127.26.226.61:43543,127.26.226.60:33887 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:56:35.657680 6117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:35.660439 6453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:35.665872 6454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:35.667835 6458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:35.668421 6117 server_base.cc:1034] running on GCE node
I20260812 06:56:35.668838 6117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:35.669605 6117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:35.670840 6117 hybrid_clock.cc:648] HybridClock initialized: now 1786517795670793 us; error 37 us; skew 500 ppm
I20260812 06:56:35.671145 6117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:35 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517795, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:35.673800 6117 init.cc:377] Logged in from keytab as oryx/127.26.226.60@KRBTEST.COM (short username oryx)
I20260812 06:56:35.675282 6117 webserver.cc:460] Webserver started at http://127.26.226.60:42029/ using document root <none> and password file <none>
I20260812 06:56:35.675760 6117 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:35.675935 6117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:35.679751 6117 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:35.682197 6476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:35.682690 6117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260812 06:56:35.682893 6117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal
uuid: "e2e89365619e4408b2ad53aee78acff8"
format_stamp: "Formatted at 2026-08-12 06:56:35 on dist-test-slave-t3q3"
server_key: "bc20a016fd25c6e576531d0d97db02e1"
server_key_iv: "cdb73f2faed6d70bfe572276c17d3504"
server_key_version: "encryptionkey@0"
I20260812 06:56:35.683214 6117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:35.697777 6117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:35.702330 6117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:35.728535 6117 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.60:33887
I20260812 06:56:35.728597 6560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.60:33887 every 8 connection(s)
I20260812 06:56:35.730042 6117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
I20260812 06:56:35.736001 6561 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:56:35.739849 6561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Bootstrap starting.
I20260812 06:56:35.758718 6561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:56:35.759292 6561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Bootstrap complete.
I20260812 06:56:35.760476 6561 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } }
I20260812 06:56:35.760777 6561 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e2e89365619e4408b2ad53aee78acff8, State: Initialized, Role: LEARNER
I20260812 06:56:35.761075 6561 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } }
I20260812 06:56:35.761950 6566 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:35.762306 6561 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:35.762537 6566 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: This master's current role is: LEARNER
I20260812 06:56:35.776615 6588 catalog_manager.cc:1259] Loaded cluster ID: 993356870845481c94b7918d6e35d6eb
I20260812 06:56:35.776743 6588 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: loading cluster ID for follower catalog manager: success
I20260812 06:56:35.780268 6588 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: acquiring CA information for follower catalog manager: success
I20260812 06:56:35.782954 6588 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:56:36.039819 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.032705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36662 (local address 127.26.226.60:33887)
0812 06:56:36.033581 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:36.033586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:36.033604 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:56:36.033669 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:36.033699 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:36.033798 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:36.033904 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:36.034468 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.035491 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.037894 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.038791 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.038937 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.039147 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:36.039410 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 06:56:36.039536 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":617,"threads_started":1}
I20260812 06:56:36.039932 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.032548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:33887 (local address 127.0.0.1:36662)
0812 06:56:36.033199 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:36.033223 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:36.033401 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:36.033956 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:36.033961 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:36.033997 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:36.034255 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.034268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.035663 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.035669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.037721 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.037738 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.038999 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.039027 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.039240 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.039283 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 06:56:36.039387 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 06:56:36.039567 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":382,"threads_started":1}
I20260812 06:56:36.041046 6530 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 06:56:36.041702 6566 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4a6c926fc984692809b846da0948692. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:36.042024 6566 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: This master's current role is: LEARNER
W20260812 06:56:36.055837 6462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:36.056392 6462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:36.056581 6462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:36.064383 6462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:36.064534 6462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:36.064728 6462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:36.074707 6462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab
--principal=oryx/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:42641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=44991
--tserver_master_addrs=127.26.226.62:36667,127.26.226.61:43543,127.26.226.60:33887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:36.076164 6462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:36.077473 6462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:36.091358 6598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:36.091598 6600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:36.091885 6597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:36.092378 6462 server_base.cc:1034] running on GCE node
I20260812 06:56:36.093089 6462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:36.094167 6462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:36.095382 6462 hybrid_clock.cc:648] HybridClock initialized: now 1786517796095339 us; error 36 us; skew 500 ppm
I20260812 06:56:36.095804 6462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:36 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517796, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:36.099910 6462 init.cc:377] Logged in from keytab as oryx/127.26.226.1@KRBTEST.COM (short username oryx)
I20260812 06:56:36.101475 6462 webserver.cc:460] Webserver started at http://127.26.226.1:44991/ using document root <none> and password file <none>
I20260812 06:56:36.102259 6462 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:36.102396 6462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:36.108704 6462 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:56:36.111570 6617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:36.113099 6462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:36.113243 6462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174"
format_stamp: "Formatted at 2026-08-12 06:56:32 on dist-test-slave-t3q3"
server_key: "3bc34aa9c50cc6034b984a6584862dd6"
server_key_iv: "4506458595526437675a65f9f34100ce"
server_key_version: "encryptionkey@0"
I20260812 06:56:36.114310 6462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:36.157567 6462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:36.162067 6462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:36.162339 6462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:36.163434 6462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:36.165062 6462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:36.165162 6462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:36.165248 6462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:36.165352 6462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:36.203279 6462 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:42641
I20260812 06:56:36.203338 6800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:42641 every 8 connection(s)
I20260812 06:56:36.205086 6462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:36.212109 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 6462
I20260812 06:56:36.212493 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 5518
Aug 12 06:56:36 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517796, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.60@KRBTEST.COM
Aug 12 06:56:36 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517796, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.61@KRBTEST.COM
Aug 12 06:56:36 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517796, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:36.239037 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.208305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:57425 (local address 127.26.226.60:33887)
0812 06:56:36.209467 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:36.209477 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:36.210229 (+ 752us) server_negotiation.cc:407] Connection header received
0812 06:56:36.213736 (+ 3507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:36.213743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:36.213828 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:36.213909 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:36.217569 (+ 3660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.219105 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.222942 (+ 3837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.223887 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.224020 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.230675 (+ 6655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:36.230682 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:36.230687 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:36.230735 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:36.234476 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.235554 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.235560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.235563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.235749 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.236454 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.236460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.236463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.236874 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:36.237142 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:36.238285 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0812 06:56:36.238602 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":985}
I20260812 06:56:36.239130 6809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.208222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:46659 (local address 127.26.226.61:43543)
0812 06:56:36.208813 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:36.208821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:36.210234 (+ 1413us) server_negotiation.cc:407] Connection header received
0812 06:56:36.213557 (+ 3323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:36.213567 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:36.213690 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:36.213849 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:36.217584 (+ 3735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.219102 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.222854 (+ 3752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.223803 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.223964 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.231308 (+ 7344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:36.231317 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:36.231322 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:36.231367 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:36.235150 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.236175 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.236181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.236184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.236340 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.237327 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.237332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.237335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.237739 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:36.237893 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:36.238473 (+ 580us) server_negotiation.cc:299] Negotiation successful
0812 06:56:36.238715 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":298,"threads_started":1}
Aug 12 06:56:36 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.2@KRBTEST.COM
I20260812 06:56:36.242069 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.208931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:43543 (local address 127.26.226.1:46659)
0812 06:56:36.210034 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:36.210094 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:36.213176 (+ 3082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:36.214512 (+ 1336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:36.214530 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:36.215877 (+ 1347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:36.217307 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.217336 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.219394 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.219402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.222701 (+ 3299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.222717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.225532 (+ 2815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.225538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.225702 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.227655 (+ 1953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:36.227684 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:36.231027 (+ 3343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:36.235336 (+ 4309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.235343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.235350 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.235986 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.236981 (+ 995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.236990 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.236994 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.237181 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.237982 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:36.237986 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:36.238149 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 06:56:36.239407 (+ 1258us) client_negotiation.cc:240] Negotiation successful
0812 06:56:36.240440 (+ 1033us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"mutex_wait_us":538,"spinlock_wait_cycles":54528,"thread_start_us":460,"threads_started":1}
I20260812 06:56:36.242094 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.208931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:33887 (local address 127.26.226.1:57425)
0812 06:56:36.210093 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:36.210117 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:36.213176 (+ 3059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:36.214512 (+ 1336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:36.214530 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:36.215876 (+ 1346us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:36.217304 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.217339 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.219402 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.219410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.222489 (+ 3079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.222511 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.224531 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.224541 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.224972 (+ 431us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.225986 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:36.226023 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:36.230387 (+ 4364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:36.234693 (+ 4306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.234702 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.234722 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.235395 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.235970 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.235975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.235978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.236309 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.237274 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:36.237283 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:36.237780 (+ 497us) client_negotiation.cc:769] Sending connection context
0812 06:56:36.239406 (+ 1626us) client_negotiation.cc:240] Negotiation successful
0812 06:56:36.240607 (+ 1201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":692,"spinlock_wait_cycles":155392,"thread_start_us":425,"threads_started":1}
I20260812 06:56:36.243804 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.210796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:57559 (local address 127.26.226.62:36667)
0812 06:56:36.211260 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:36.211266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:36.211740 (+ 474us) server_negotiation.cc:407] Connection header received
0812 06:56:36.216930 (+ 5190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:36.219880 (+ 2950us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:36.220029 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:36.226057 (+ 6028us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:36.226303 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.227366 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.229563 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.230016 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.230164 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.234568 (+ 4404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:36.234574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:36.234582 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:36.234629 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:36.238815 (+ 4186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.240336 (+ 1521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.240342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.240346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.240517 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.241109 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.241115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.241119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.241531 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:36.241679 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:36.243094 (+ 1415us) server_negotiation.cc:299] Negotiation successful
0812 06:56:36.243345 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":332,"threads_started":1}
I20260812 06:56:36.244042 6805 heartbeater.cc:347] Connected to a master server at 127.26.226.61:43543
I20260812 06:56:36.244266 6804 heartbeater.cc:347] Connected to a master server at 127.26.226.60:33887
I20260812 06:56:36.242885 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.210971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.1:57559)
0812 06:56:36.211623 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:36.211638 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:36.213236 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:36.222530 (+ 9294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:36.222538 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:36.224834 (+ 2296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:36.225309 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.225323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.227519 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.227528 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.229354 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.229371 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.230244 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.230252 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.230465 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.231096 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:36.231120 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:36.234318 (+ 3198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:36.239019 (+ 4701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.239024 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.239028 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.240208 (+ 1180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.240654 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.240659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.240663 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.240859 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.241754 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:36.241761 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:36.241960 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 06:56:36.242130 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 06:56:36.242426 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":354,"threads_started":1}
I20260812 06:56:36.244449 6804 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:36.244470 6805 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:36.245693 6802 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36667
I20260812 06:56:36.245759 6804 heartbeater.cc:511] Master 127.26.226.60:33887 requested a full tablet report, sending...
I20260812 06:56:36.245790 6805 heartbeater.cc:511] Master 127.26.226.61:43543 requested a full tablet report, sending...
I20260812 06:56:36.245855 6802 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:36.246351 6802 heartbeater.cc:511] Master 127.26.226.62:36667 requested a full tablet report, sending...
I20260812 06:56:36.247612 5210 ts_manager.cc:194] Re-registered known tserver with Master: 37919c2b404d4ac4b9d7e5f2e4ec0174 (127.26.226.1:42641)
I20260812 06:56:36.248471 6513 ts_manager.cc:194] Registered new tserver with Master: 37919c2b404d4ac4b9d7e5f2e4ec0174 (127.26.226.1:42641)
I20260812 06:56:36.248564 5210 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:57559
I20260812 06:56:36.250275 6180 ts_manager.cc:194] Registered new tserver with Master: 37919c2b404d4ac4b9d7e5f2e4ec0174 (127.26.226.1:42641)
I20260812 06:56:36.251173 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.213806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42950 (local address 127.26.226.2:33865)
0812 06:56:36.226705 (+ 12899us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:36.226714 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:36.226744 (+ 30us) server_negotiation.cc:407] Connection header received
0812 06:56:36.226812 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:36.226821 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:36.227084 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:36.227239 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:36.229456 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.231358 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.234774 (+ 3416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.235832 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.235960 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.242575 (+ 6615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:36.242591 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:36.242609 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:36.242666 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:36.247692 (+ 5026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.248724 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.248732 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.248738 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.248904 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.249385 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.249392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.249395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.249753 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:36.250091 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:36.250343 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 06:56:36.250616 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12712,"thread_start_us":12551,"threads_started":1}
I20260812 06:56:36.253203 5721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:36.337718 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 5518
I20260812 06:56:36.366819 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:33865
--local_ip_for_outbound_sockets=127.26.226.2
--tserver_master_addrs=127.26.226.62:36667
--webserver_port=42359
--webserver_interface=127.26.226.2
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab
--principal=oryx/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.226.62:36667,127.26.226.61:43543
--tserver_master_addrs=127.26.226.62:36667,127.26.226.61:43543,127.26.226.60:33887 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:56:36.643294 6425 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: NON_VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: true } } }
I20260812 06:56:36.643733 6425 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:36.645344 6825 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692: attempting to promote NON_VOTER e2e89365619e4408b2ad53aee78acff8 to VOTER
I20260812 06:56:36.646153 6825 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } }
I20260812 06:56:36.648679 6197 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 FOLLOWER]: Refusing update from remote peer f4a6c926fc984692809b846da0948692: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 06:56:36.648765 6530 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 LEARNER]: Refusing update from remote peer f4a6c926fc984692809b846da0948692: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 06:56:36.649364 6425 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:56:36.649760 6823 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:56:36.658061 6425 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e2e89365619e4408b2ad53aee78acff8 (127.26.226.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:36.659471 6530 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e2e89365619e4408b2ad53aee78acff8 (127.26.226.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:36.660458 6197 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e2e89365619e4408b2ad53aee78acff8 (127.26.226.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:36.662109 6825 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:36.662410 6825 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:36.662751 6825 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:36.662998 6825 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:36.663311 6825 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:36.663522 6825 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:36.665551 6833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:36.666101 6833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:56:36.666455 6830 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4a6c926fc984692809b846da0948692" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:36.666827 6830 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 06:56:36.735451 6822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:36.735898 6822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:36.736132 6822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:36.751050 6822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:36.751209 6822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:36.751456 6822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:36.767810 6822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab
--principal=oryx/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:33865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=42359
--tserver_master_addrs=127.26.226.62:36667,127.26.226.61:43543,127.26.226.60:33887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:36.769838 6822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:36.771735 6822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:36.787703 6840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:36.787842 6839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:36.787721 6843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:36.788367 6822 server_base.cc:1034] running on GCE node
I20260812 06:56:36.789050 6822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:36.789989 6822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:36.791240 6822 hybrid_clock.cc:648] HybridClock initialized: now 1786517796791186 us; error 45 us; skew 500 ppm
I20260812 06:56:36.791579 6822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:36 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517796, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:36.794907 6822 init.cc:377] Logged in from keytab as oryx/127.26.226.2@KRBTEST.COM (short username oryx)
I20260812 06:56:36.796595 6822 webserver.cc:460] Webserver started at http://127.26.226.2:42359/ using document root <none> and password file <none>
I20260812 06:56:36.797355 6822 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:36.797526 6822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:36.803079 6822 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:56:36.805896 6852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:36.807263 6822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:36.807446 6822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "20b470bb7e1a457eba01ba8453cf2a7e"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-t3q3"
server_key: "59265eef7d4c62bccd3b6f3b9b3955d4"
server_key_iv: "2d28b22bf8a9869b79d57896dda294ee"
server_key_version: "encryptionkey@0"
I20260812 06:56:36.808566 6822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:36.822257 6822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:36.826635 6822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:36.826972 6822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:36.828054 6822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:36.829457 6822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:36.829572 6822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:36.829701 6822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:36.829800 6822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:36.865108 6822 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33865
I20260812 06:56:36.865221 7044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33865 every 8 connection(s)
I20260812 06:56:36.866717 6822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:36.869565 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 6822
I20260812 06:56:36.869884 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 5741
Aug 12 06:56:36 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.3@KRBTEST.COM
Aug 12 06:56:36 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517796, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
Aug 12 06:56:36 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517796, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.61@KRBTEST.COM
Aug 12 06:56:36 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517796, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.60@KRBTEST.COM
I20260812 06:56:36.909614 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.871682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48406 (local address 127.26.226.3:46431)
0812 06:56:36.872481 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:36.872494 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:36.882125 (+ 9631us) server_negotiation.cc:407] Connection header received
0812 06:56:36.882434 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:36.882451 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:36.882831 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:36.883063 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:36.885984 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.887650 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.889786 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.890665 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.890807 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.898766 (+ 7959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:36.898789 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:36.898813 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:36.898868 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:36.904916 (+ 6048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.905895 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.905910 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.905914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.906103 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.906675 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.906682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.906686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.907184 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:36.907508 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:36.908841 (+ 1333us) server_negotiation.cc:299] Negotiation successful
0812 06:56:36.909156 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":363,"threads_started":1}
I20260812 06:56:36.910229 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.870000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:38545 (local address 127.26.226.60:33887)
0812 06:56:36.871095 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:36.871100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:36.884619 (+ 13519us) server_negotiation.cc:407] Connection header received
0812 06:56:36.886575 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:36.886581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:36.886674 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:36.886793 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:36.889810 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.890935 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.894479 (+ 3544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.895273 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.895388 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.901824 (+ 6436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:36.901834 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:36.901838 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:36.901885 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:36.905635 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.907014 (+ 1379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.907020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.907023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.907174 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.907775 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.907780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.907783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.908173 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:36.908318 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:36.909500 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0812 06:56:36.909834 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":982,"thread_start_us":849,"threads_started":1}
I20260812 06:56:36.910229 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.871056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:59325 (local address 127.26.226.62:36667)
0812 06:56:36.871598 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:36.871607 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:36.883792 (+ 12185us) server_negotiation.cc:407] Connection header received
0812 06:56:36.886575 (+ 2783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:36.886581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:36.886675 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:36.886793 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:36.889814 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.890963 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.894492 (+ 3529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.895194 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.895284 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.901106 (+ 5822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:36.901115 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:36.901121 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:36.901181 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:36.906019 (+ 4838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.907038 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.907044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.907046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.907230 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.907786 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.907792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.907795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.908181 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:36.908343 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:36.909567 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0812 06:56:36.909835 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":835,"threads_started":1}
I20260812 06:56:36.910266 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.884839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:56107 (local address 127.26.226.61:43543)
0812 06:56:36.885407 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:36.885415 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:36.886103 (+ 688us) server_negotiation.cc:407] Connection header received
0812 06:56:36.887144 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:36.887149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:36.887242 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:36.887334 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:36.889811 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.890985 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.894490 (+ 3505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.895170 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.895282 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.901606 (+ 6324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:36.901615 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:36.901622 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:36.901685 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:36.905564 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.907394 (+ 1830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.907400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.907403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.907555 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:36.908096 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:36.908103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:36.908106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:36.908544 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:36.908692 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:36.909617 (+ 925us) server_negotiation.cc:299] Negotiation successful
0812 06:56:36.909860 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":303,"threads_started":1}
I20260812 06:56:36.911494 5931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:36.913031 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.870562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.2:59325)
0812 06:56:36.883230 (+ 12668us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:36.883285 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:36.886113 (+ 2828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:36.887102 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:36.887114 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:36.888352 (+ 1238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:36.889583 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.889614 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.891181 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.891187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.894299 (+ 3112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.894316 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.895685 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.895694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.896195 (+ 501us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.897416 (+ 1221us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:36.897442 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:36.900793 (+ 3351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:36.906212 (+ 5419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.906218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.906221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.906870 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.907388 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.907392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.907395 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.907648 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.908430 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:36.908442 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:36.909001 (+ 559us) client_negotiation.cc:769] Sending connection context
0812 06:56:36.910518 (+ 1517us) client_negotiation.cc:240] Negotiation successful
0812 06:56:36.911177 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12289,"mutex_wait_us":692,"spinlock_wait_cycles":26240,"thread_start_us":13361,"threads_started":1}
I20260812 06:56:36.915048 7045 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36667
I20260812 06:56:36.915372 7045 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:36.916286 7045 heartbeater.cc:511] Master 127.26.226.62:36667 requested a full tablet report, sending...
I20260812 06:56:36.916986 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.885405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:43543 (local address 127.26.226.2:56107)
0812 06:56:36.885981 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:36.885997 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:36.886983 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:36.887624 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:36.887630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:36.888338 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:36.889583 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.889614 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.891181 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.891187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.894299 (+ 3112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.894316 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.895685 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.895696 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.896243 (+ 547us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.897620 (+ 1377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:36.897643 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:36.901368 (+ 3725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:36.905778 (+ 4410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.905789 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.905809 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.907250 (+ 1441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.907726 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.907731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.907737 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.907918 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.908781 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:36.908785 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:36.909002 (+ 217us) client_negotiation.cc:769] Sending connection context
0812 06:56:36.910518 (+ 1516us) client_negotiation.cc:240] Negotiation successful
0812 06:56:36.916485 (+ 5967us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":6247,"spinlock_wait_cycles":42240,"thread_start_us":312,"threads_started":1}
I20260812 06:56:36.917083 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:36.871117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:33887 (local address 127.26.226.2:38545)
0812 06:56:36.884485 (+ 13368us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:36.884509 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:36.886116 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:36.887106 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:36.887116 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:36.888361 (+ 1245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:36.889586 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.889614 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.891164 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.891168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.894254 (+ 3086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:36.894277 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:36.895737 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:36.895746 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:36.896229 (+ 483us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:36.897174 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:36.897231 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:36.901571 (+ 4340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:36.905818 (+ 4247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.905824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.905830 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.906858 (+ 1028us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.907341 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:36.907346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:36.907352 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:36.907627 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:36.908430 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:36.908438 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:36.909001 (+ 563us) client_negotiation.cc:769] Sending connection context
0812 06:56:36.910514 (+ 1513us) client_negotiation.cc:240] Negotiation successful
0812 06:56:36.916621 (+ 6107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13205,"mutex_wait_us":6131,"spinlock_wait_cycles":149632,"thread_start_us":11610,"threads_started":1}
I20260812 06:56:36.918031 5210 ts_manager.cc:194] Re-registered known tserver with Master: 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865)
I20260812 06:56:36.918165 7051 heartbeater.cc:347] Connected to a master server at 127.26.226.61:43543
I20260812 06:56:36.918294 7051 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:36.918694 7051 heartbeater.cc:511] Master 127.26.226.61:43543 requested a full tablet report, sending...
I20260812 06:56:36.919198 5210 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:59325
I20260812 06:56:36.919814 6180 ts_manager.cc:194] Registered new tserver with Master: 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865)
I20260812 06:56:36.919993 7048 heartbeater.cc:347] Connected to a master server at 127.26.226.60:33887
I20260812 06:56:36.920158 7048 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:36.920687 7048 heartbeater.cc:511] Master 127.26.226.60:33887 requested a full tablet report, sending...
I20260812 06:56:36.922019 6513 ts_manager.cc:194] Registered new tserver with Master: 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865)
I20260812 06:56:36.998812 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 5741
I20260812 06:56:37.024858 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:46431
--local_ip_for_outbound_sockets=127.26.226.3
--tserver_master_addrs=127.26.226.62:36667
--webserver_port=44371
--webserver_interface=127.26.226.3
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab
--principal=oryx/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.226.62:36667,127.26.226.61:43543
--tserver_master_addrs=127.26.226.62:36667,127.26.226.61:43543,127.26.226.60:33887 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:56:37.252375 6802 heartbeater.cc:503] Master 127.26.226.62:36667 was elected leader, sending a full tablet report...
W20260812 06:56:37.379007 7065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:37.379297 7065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:37.379396 7065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:37.386600 7065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:37.386706 7065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:37.386826 7065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:37.396471 7065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab
--principal=oryx/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:46431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=44371
--tserver_master_addrs=127.26.226.62:36667,127.26.226.61:43543,127.26.226.60:33887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:37.397866 7065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:37.399534 7065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:37.413691 7074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:37.413748 7076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:37.413800 7073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:37.414741 7065 server_base.cc:1034] running on GCE node
I20260812 06:56:37.415553 7065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:37.416682 7065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:37.417929 7065 hybrid_clock.cc:648] HybridClock initialized: now 1786517797417861 us; error 65 us; skew 500 ppm
I20260812 06:56:37.418264 7065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:37 dist-test-slave-t3q3 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517797, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:37.421711 7065 init.cc:377] Logged in from keytab as oryx/127.26.226.3@KRBTEST.COM (short username oryx)
I20260812 06:56:37.423295 7065 webserver.cc:460] Webserver started at http://127.26.226.3:44371/ using document root <none> and password file <none>
I20260812 06:56:37.424046 7065 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:37.424278 7065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:37.430007 7065 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260812 06:56:37.433041 7090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:37.434574 7065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:37.434763 7065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "c79032ca1b154df28ff4759562daa6e5"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-t3q3"
server_key: "408e1f56c344e05be1af958c19961216"
server_key_iv: "49575fe07e48dd4e112a6af4e8988ffa"
server_key_version: "encryptionkey@0"
I20260812 06:56:37.435810 7065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:37.453317 7065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:37.457964 7065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:37.458269 7065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:37.459328 7065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:37.460975 7065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:37.461102 7065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:37.461232 7065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:37.461326 7065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:37.496763 7065 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:46431
I20260812 06:56:37.496827 7281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:46431 every 8 connection(s)
I20260812 06:56:37.498363 7065 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:37.503612 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 7065
Aug 12 06:56:37 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517797, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.60@KRBTEST.COM
Aug 12 06:56:37 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517797, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.61@KRBTEST.COM
Aug 12 06:56:37 dist-test-slave-t3q3 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517797, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:37.553844 7291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.516089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:43543 (local address 127.26.226.3:47129)
0812 06:56:37.519637 (+ 3548us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:37.519658 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:37.519784 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:37.521754 (+ 1970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:37.521759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:37.522381 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:37.522825 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.522838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.531085 (+ 8247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.531091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.533072 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.533094 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.535322 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.535331 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.535607 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.536401 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:37.536432 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:37.540839 (+ 4407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:37.548421 (+ 7582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:37.548433 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:37.548452 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:37.549135 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:37.549680 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:37.549686 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:37.549691 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:37.549972 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:37.550813 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:37.550823 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:37.551326 (+ 503us) client_negotiation.cc:769] Sending connection context
0812 06:56:37.551687 (+ 361us) client_negotiation.cc:240] Negotiation successful
0812 06:56:37.552311 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3362,"thread_start_us":764,"threads_started":1}
I20260812 06:56:37.555039 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.513379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:47129 (local address 127.26.226.61:43543)
0812 06:56:37.521347 (+ 7968us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.521356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.521381 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:56:37.521454 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.521462 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.521572 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.521715 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:37.523024 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.524116 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.533803 (+ 9687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.534776 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.534913 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.541863 (+ 6950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:37.541873 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:37.541881 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:37.541942 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:37.547745 (+ 5803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.549316 (+ 1571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.549326 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.549336 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.549524 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.550127 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.550134 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.550141 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.550573 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:37.550716 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.554389 (+ 3673us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.554615 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7813,"thread_start_us":7680,"threads_started":1}
I20260812 06:56:37.556206 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.502101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.3:46257)
0812 06:56:37.516648 (+ 14547us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:37.516669 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:37.516903 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:37.517903 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:37.517919 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:37.519229 (+ 1310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:37.520125 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.520150 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.522277 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.522283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.530393 (+ 8110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.530414 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.533355 (+ 2941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.533366 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.534370 (+ 1004us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.541074 (+ 6704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:37.541095 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:37.545350 (+ 4255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:37.551229 (+ 5879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:37.551234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:37.551238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:37.551837 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:37.553625 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:37.553632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:37.553636 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:37.553831 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:37.554916 (+ 1085us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:37.554921 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:37.555082 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 06:56:37.555483 (+ 401us) client_negotiation.cc:240] Negotiation successful
0812 06:56:37.555779 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14388,"mutex_wait_us":610,"thread_start_us":9411,"threads_started":1}
I20260812 06:56:37.556365 7290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.501268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:46257 (local address 127.26.226.62:36667)
0812 06:56:37.508580 (+ 7312us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.508590 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.516769 (+ 8179us) server_negotiation.cc:407] Connection header received
0812 06:56:37.517071 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.517080 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.517186 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.517290 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:37.520310 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.521353 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.530563 (+ 9210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.531329 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.531460 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.545599 (+ 14139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:37.545609 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:37.545615 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:37.545664 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:37.551065 (+ 5401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.551967 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.551973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.551976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.552160 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.554284 (+ 2124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.554289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.554292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.554635 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:37.554799 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.555637 (+ 838us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.555915 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7116,"thread_start_us":7361,"threads_started":1}
I20260812 06:56:37.556344 7288 heartbeater.cc:347] Connected to a master server at 127.26.226.61:43543
I20260812 06:56:37.556874 7288 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:37.557813 7288 heartbeater.cc:511] Master 127.26.226.61:43543 requested a full tablet report, sending...
I20260812 06:56:37.558713 7295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.502101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:33887 (local address 127.26.226.3:48581)
0812 06:56:37.512580 (+ 10479us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:37.512644 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:37.516201 (+ 3557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:37.518678 (+ 2477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:37.518685 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:37.520186 (+ 1501us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:37.520515 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.520529 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.521957 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.521968 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.524251 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.524267 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.533356 (+ 9089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.533368 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.533822 (+ 454us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.534711 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:37.534741 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:37.539952 (+ 5211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:37.553526 (+ 13574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:37.553531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:37.553534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:37.554135 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:37.555210 (+ 1075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:37.555214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:37.555219 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:37.555403 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:37.557511 (+ 2108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:37.557516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:37.557733 (+ 217us) client_negotiation.cc:769] Sending connection context
0812 06:56:37.557863 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 06:56:37.558244 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10160,"mutex_wait_us":22,"thread_start_us":16961,"threads_started":1}
I20260812 06:56:37.559818 6180 ts_manager.cc:194] Registered new tserver with Master: c79032ca1b154df28ff4759562daa6e5 (127.26.226.3:46431)
I20260812 06:56:37.560454 7285 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36667
I20260812 06:56:37.560576 7285 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:37.561033 7292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.502505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:48581 (local address 127.26.226.60:33887)
0812 06:56:37.513152 (+ 10647us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.513159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.513182 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:56:37.518111 (+ 4929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.518121 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.518245 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.518372 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:37.520691 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.521748 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.529805 (+ 8057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.530682 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.530810 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.540474 (+ 9664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:37.540485 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:37.540494 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:37.540551 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:37.553345 (+ 12794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.554894 (+ 1549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.554900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.554903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.555057 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.556872 (+ 1815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.556878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.556883 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.557335 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:37.560187 (+ 2852us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.560322 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.560597 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10496,"thread_start_us":11245,"threads_started":1}
I20260812 06:56:37.561012 7285 heartbeater.cc:511] Master 127.26.226.62:36667 requested a full tablet report, sending...
I20260812 06:56:37.562384 5210 ts_manager.cc:194] Re-registered known tserver with Master: c79032ca1b154df28ff4759562daa6e5 (127.26.226.3:46431)
I20260812 06:56:37.562525 7287 heartbeater.cc:347] Connected to a master server at 127.26.226.60:33887
I20260812 06:56:37.562649 7287 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:37.563100 7287 heartbeater.cc:511] Master 127.26.226.60:33887 requested a full tablet report, sending...
I20260812 06:56:37.563833 5210 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:46257
I20260812 06:56:37.564447 6513 ts_manager.cc:194] Registered new tserver with Master: c79032ca1b154df28ff4759562daa6e5 (127.26.226.3:46431)
I20260812 06:56:37.571388 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.523659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.26.226.62:36667)
0812 06:56:37.548396 (+ 24737us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.548402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.548423 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:56:37.548484 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.548489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.548584 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.548700 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:37.552085 (+ 3385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.553432 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.557889 (+ 4457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.558583 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.558711 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.565449 (+ 6738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:37.565456 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:37.565460 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:37.565498 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:37.568602 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.569310 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.569315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.569317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.569441 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.569895 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.569900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.569903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.570227 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:37.570479 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.570842 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.571052 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24599,"thread_start_us":25260,"threads_started":1}
I20260812 06:56:37.573097 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.534729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33216 (local address 127.26.226.61:43543)
0812 06:56:37.550568 (+ 15839us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.550576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.554224 (+ 3648us) server_negotiation.cc:407] Connection header received
0812 06:56:37.554482 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.554488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.554579 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.554685 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:37.557946 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.559127 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.561157 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.562168 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.562302 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.566171 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:37.566180 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:37.566186 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:37.566238 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:37.569950 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.570784 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.570792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.570796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.570963 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.571461 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.571466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.571468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.571849 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:37.572150 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.572455 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.572663 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15675,"thread_start_us":15849,"threads_started":1}
I20260812 06:56:37.576843 7311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.527895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.26.226.60:33887)
0812 06:56:37.548296 (+ 20401us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.548303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.548781 (+ 478us) server_negotiation.cc:407] Connection header received
0812 06:56:37.551576 (+ 2795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.551583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.551670 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.551760 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:37.556550 (+ 4790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.560656 (+ 4106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.564513 (+ 3857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.565377 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.565493 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.569787 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:37.569796 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:37.569802 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:37.569845 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:37.574013 (+ 4168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.574895 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.574900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.574904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.575085 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.575521 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.575525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.575527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.575801 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:37.575972 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.576395 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.576555 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17445,"thread_start_us":20167,"threads_started":1}
I20260812 06:56:37.575832 5210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:37.583375 5210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:37.616008 7337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.603303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.26.226.3:46431)
0812 06:56:37.606380 (+ 3077us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.606391 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.606436 (+ 45us) server_negotiation.cc:407] Connection header received
0812 06:56:37.606611 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.606626 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.606935 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.608041 (+ 1106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:37.608709 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.610514 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.613561 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.615077 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.615219 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.615406 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.615646 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.615766 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2906,"thread_start_us":2717,"threads_started":1}
I20260812 06:56:37.616199 7331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.602977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:46431 (local address 127.0.0.1:48408)
0812 06:56:37.603690 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:37.603711 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:37.603851 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:37.607202 (+ 3351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:37.607210 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:37.607241 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:37.607706 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.607720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.610689 (+ 2969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.610697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.613035 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.613050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.615278 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.615285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.615467 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.615502 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 06:56:37.615609 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 06:56:37.615751 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":1,"thread_start_us":348,"threads_started":1}
I20260812 06:56:37.617692 7346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.604142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.26.226.1:42641)
0812 06:56:37.609590 (+ 5448us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.609598 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.609623 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:56:37.609710 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.609716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.609907 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.610111 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:37.610782 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.612401 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.614941 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.616408 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.616549 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.616739 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.617165 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.617319 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5277,"thread_start_us":5171,"threads_started":1}
I20260812 06:56:37.618044 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.603776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:42641 (local address 127.0.0.1:35634)
0812 06:56:37.604405 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:37.604423 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:37.604550 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:37.610114 (+ 5564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:37.610117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:37.610153 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:37.610556 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.610571 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.612599 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.612605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.614783 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.614798 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.616852 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.616857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.616978 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.617000 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:56:37.617104 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 06:56:37.617256 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":126,"thread_start_us":370,"threads_started":1}
I20260812 06:56:37.618993 7345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.603881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33865 (local address 127.0.0.1:42956)
0812 06:56:37.608214 (+ 4333us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:37.608235 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:37.608350 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:37.610314 (+ 1964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:37.610319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:37.610348 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:37.610934 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.610946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.612395 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.612401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.615072 (+ 2671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.615086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.618201 (+ 3115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.618207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.618398 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.618432 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:56:37.618531 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 06:56:37.618656 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4156,"mutex_wait_us":63,"thread_start_us":4010,"threads_started":1}
I20260812 06:56:37.619817 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.604482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42956 (local address 127.26.226.2:33865)
0812 06:56:37.606727 (+ 2245us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.606735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.608473 (+ 1738us) server_negotiation.cc:407] Connection header received
0812 06:56:37.608597 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.608605 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.608822 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.609037 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:37.611137 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.612264 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.616273 (+ 4009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.617701 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.617830 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.618018 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.619378 (+ 1360us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.619549 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2099,"thread_start_us":1980,"threads_started":1}
I20260812 06:56:37.620577 7171 tablet_service.cc:1467] Processing CreateTablet for tablet c0bad8b01f6041c4ba75f16fa20a2fed (DEFAULT_TABLE table=test-table [id=105cfa474612422ebfe632fd93c3fcf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:37.621192 6687 tablet_service.cc:1467] Processing CreateTablet for tablet c0bad8b01f6041c4ba75f16fa20a2fed (DEFAULT_TABLE table=test-table [id=105cfa474612422ebfe632fd93c3fcf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:37.622156 7171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0bad8b01f6041c4ba75f16fa20a2fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:37.622584 6687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0bad8b01f6041c4ba75f16fa20a2fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:37.623198 6939 tablet_service.cc:1467] Processing CreateTablet for tablet c0bad8b01f6041c4ba75f16fa20a2fed (DEFAULT_TABLE table=test-table [id=105cfa474612422ebfe632fd93c3fcf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:37.624562 6939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0bad8b01f6041c4ba75f16fa20a2fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:37.634662 7348 tablet_bootstrap.cc:492] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: Bootstrap starting.
I20260812 06:56:37.638480 7348 tablet_bootstrap.cc:654] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:37.639734 7348 log.cc:824] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:37.640992 7356 tablet_bootstrap.cc:492] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: Bootstrap starting.
I20260812 06:56:37.644820 7353 tablet_bootstrap.cc:492] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: Bootstrap starting.
I20260812 06:56:37.646068 7356 tablet_bootstrap.cc:654] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:37.647842 7356 log.cc:824] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:37.648013 7353 tablet_bootstrap.cc:654] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:37.649360 7353 log.cc:824] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:37.651051 7356 tablet_bootstrap.cc:492] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: No bootstrap required, opened a new log
I20260812 06:56:37.651041 7348 tablet_bootstrap.cc:492] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: No bootstrap required, opened a new log
I20260812 06:56:37.651329 7348 ts_tablet_manager.cc:1397] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260812 06:56:37.651330 7356 ts_tablet_manager.cc:1397] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260812 06:56:37.651937 7353 tablet_bootstrap.cc:492] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: No bootstrap required, opened a new log
I20260812 06:56:37.652236 7353 ts_tablet_manager.cc:1397] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 06:56:37.659709 7353 raft_consensus.cc:348] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:37.660125 7353 raft_consensus.cc:374] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:37.660235 7353 raft_consensus.cc:729] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37919c2b404d4ac4b9d7e5f2e4ec0174, State: Initialized, Role: FOLLOWER
I20260812 06:56:37.660849 7353 consensus_queue.cc:260] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:37.662674 7353 ts_tablet_manager.cc:1428] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260812 06:56:37.662420 7348 raft_consensus.cc:348] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:37.662431 7356 raft_consensus.cc:348] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:37.662897 7348 raft_consensus.cc:374] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:37.663008 7348 raft_consensus.cc:729] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79032ca1b154df28ff4759562daa6e5, State: Initialized, Role: FOLLOWER
I20260812 06:56:37.663635 7356 raft_consensus.cc:374] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:37.663815 7356 raft_consensus.cc:729] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b470bb7e1a457eba01ba8453cf2a7e, State: Initialized, Role: FOLLOWER
I20260812 06:56:37.663694 7348 consensus_queue.cc:260] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:37.664525 7356 consensus_queue.cc:260] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:37.671247 7285 heartbeater.cc:503] Master 127.26.226.62:36667 was elected leader, sending a full tablet report...
I20260812 06:56:37.672154 7348 ts_tablet_manager.cc:1428] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: Time spent starting tablet: real 0.021s user 0.017s sys 0.005s
I20260812 06:56:37.672940 7045 heartbeater.cc:503] Master 127.26.226.62:36667 was elected leader, sending a full tablet report...
I20260812 06:56:37.675241 7356 ts_tablet_manager.cc:1428] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: Time spent starting tablet: real 0.024s user 0.015s sys 0.007s
W20260812 06:56:37.709088 6806 tablet.cc:2369] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:37.718024 7363 raft_consensus.cc:482] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:37.718578 7363 raft_consensus.cc:504] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:37.720595 7363 leader_election.cc:288] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37919c2b404d4ac4b9d7e5f2e4ec0174 (127.26.226.1:42641), 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865)
I20260812 06:56:37.726361 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.721224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:37265 (local address 127.26.226.2:33865)
0812 06:56:37.721447 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.721453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.721472 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:56:37.721543 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.721546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.721602 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.721700 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:37.722073 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.722796 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.724538 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.725373 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.725539 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.725615 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.725955 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.726086 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 06:56:37.726390 7295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.720876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33865 (local address 127.26.226.3:37265)
0812 06:56:37.721167 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:37.721192 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:37.721360 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:37.721720 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:37.721724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:37.721748 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:37.721926 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.721934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.722954 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.722959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.724392 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.724404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.725539 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.725550 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.725791 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.725820 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 06:56:37.725927 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 06:56:37.726075 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":33}
I20260812 06:56:37.726404 7346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.720981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:37165 (local address 127.26.226.1:42641)
0812 06:56:37.721276 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.721283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.721464 (+ 181us) server_negotiation.cc:407] Connection header received
0812 06:56:37.721664 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.721667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.721728 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.721832 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:37.722235 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.722928 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.724601 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.725400 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.725592 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.725670 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.726010 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.726139 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260812 06:56:37.726430 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.721050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:42641 (local address 127.26.226.3:37165)
0812 06:56:37.721365 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:37.721380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:37.721513 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:37.721854 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:37.721858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:37.721888 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:37.722064 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.722072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.723071 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.723078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.724431 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:37.724445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.725595 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.725602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:37.725852 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.725876 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:56:37.725982 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 06:56:37.726115 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":5}
I20260812 06:56:37.727514 6730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0bad8b01f6041c4ba75f16fa20a2fed" candidate_uuid: "c79032ca1b154df28ff4759562daa6e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" is_pre_election: true
I20260812 06:56:37.727512 6967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0bad8b01f6041c4ba75f16fa20a2fed" candidate_uuid: "c79032ca1b154df28ff4759562daa6e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" is_pre_election: true
I20260812 06:56:37.728128 6967 raft_consensus.cc:2454] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c79032ca1b154df28ff4759562daa6e5 in term 0.
I20260812 06:56:37.728142 6730 raft_consensus.cc:2454] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c79032ca1b154df28ff4759562daa6e5 in term 0.
I20260812 06:56:37.728911 7097 leader_election.cc:302] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20b470bb7e1a457eba01ba8453cf2a7e, c79032ca1b154df28ff4759562daa6e5; no voters:
I20260812 06:56:37.729360 7363 raft_consensus.cc:2781] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:37.729523 7363 raft_consensus.cc:482] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:37.729647 7363 raft_consensus.cc:3037] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:37.732520 7363 raft_consensus.cc:504] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:37.733251 7363 leader_election.cc:288] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [CANDIDATE]: Term 1 election: Requested vote from peers 37919c2b404d4ac4b9d7e5f2e4ec0174 (127.26.226.1:42641), 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865)
I20260812 06:56:37.733801 6730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0bad8b01f6041c4ba75f16fa20a2fed" candidate_uuid: "c79032ca1b154df28ff4759562daa6e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174"
I20260812 06:56:37.733843 6967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0bad8b01f6041c4ba75f16fa20a2fed" candidate_uuid: "c79032ca1b154df28ff4759562daa6e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20b470bb7e1a457eba01ba8453cf2a7e"
I20260812 06:56:37.734046 6730 raft_consensus.cc:3037] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:37.734068 6967 raft_consensus.cc:3037] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:37.737026 6730 raft_consensus.cc:2454] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79032ca1b154df28ff4759562daa6e5 in term 1.
I20260812 06:56:37.737517 6967 raft_consensus.cc:2454] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79032ca1b154df28ff4759562daa6e5 in term 1.
I20260812 06:56:37.737720 7095 leader_election.cc:302] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37919c2b404d4ac4b9d7e5f2e4ec0174, c79032ca1b154df28ff4759562daa6e5; no voters:
I20260812 06:56:37.738099 7363 raft_consensus.cc:2781] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:37.738708 7363 raft_consensus.cc:686] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 1 LEADER]: Becoming Leader. State: Replica: c79032ca1b154df28ff4759562daa6e5, State: Running, Role: LEADER
I20260812 06:56:37.739720 7363 consensus_queue.cc:237] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:37.745833 5210 catalog_manager.cc:5495] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c79032ca1b154df28ff4759562daa6e5 (127.26.226.3). New cstate: current_term: 1 leader_uuid: "c79032ca1b154df28ff4759562daa6e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } health_report { overall_health: UNKNOWN } } }
W20260812 06:56:37.756830 7289 tablet.cc:2369] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:37.773921 7337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.768589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.26.226.3:46431)
0812 06:56:37.768785 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.768790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.769030 (+ 240us) server_negotiation.cc:407] Connection header received
0812 06:56:37.769212 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.769216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.769310 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.769453 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:37.769762 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.770566 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.771884 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.772390 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.772505 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.772944 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:37.773468 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:37.773594 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.773643 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.773711 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:56:37.782549 6730 raft_consensus.cc:1261] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 1 FOLLOWER]: Refusing update from remote peer c79032ca1b154df28ff4759562daa6e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:37.782555 6967 raft_consensus.cc:1261] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e [term 1 FOLLOWER]: Refusing update from remote peer c79032ca1b154df28ff4759562daa6e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:37.783749 7365 consensus_queue.cc:1035] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:37.784150 7363 consensus_queue.cc:1035] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:37.822400 5209 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:37.822751 5209 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:37.830046 5209 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4a6c926fc984692809b846da0948692: Sending DeleteTablet for 3 replicas of tablet c0bad8b01f6041c4ba75f16fa20a2fed
I20260812 06:56:37.831374 7171 tablet_service.cc:1514] Processing DeleteTablet for tablet c0bad8b01f6041c4ba75f16fa20a2fed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:37 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:48408
I20260812 06:56:37.831408 6687 tablet_service.cc:1514] Processing DeleteTablet for tablet c0bad8b01f6041c4ba75f16fa20a2fed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:37 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:35634
I20260812 06:56:37.835394 6939 tablet_service.cc:1514] Processing DeleteTablet for tablet c0bad8b01f6041c4ba75f16fa20a2fed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:37 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:42956
I20260812 06:56:37.843803 7390 tablet_replica.cc:330] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: stopping tablet replica
I20260812 06:56:37.844110 7386 tablet_replica.cc:330] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: stopping tablet replica
I20260812 06:56:37.844759 7390 raft_consensus.cc:2229] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:37.844756 7386 raft_consensus.cc:2229] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:37.845459 7386 raft_consensus.cc:2258] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:37.845504 7390 raft_consensus.cc:2258] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:37.846402 7385 tablet_replica.cc:330] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: stopping tablet replica
I20260812 06:56:37.846879 7385 raft_consensus.cc:2229] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:37.847666 7385 raft_consensus.cc:2258] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:37.847725 7390 ts_tablet_manager.cc:1905] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:37.849864 7386 ts_tablet_manager.cc:1905] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:37.851630 7385 ts_tablet_manager.cc:1905] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:37.852455 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 5153
I20260812 06:56:37.853652 5263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:37.858664 7390 ts_tablet_manager.cc:1918] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:37.858870 7390 log.cc:1196] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/c0bad8b01f6041c4ba75f16fa20a2fed
I20260812 06:56:37.859617 7390 ts_tablet_manager.cc:1939] T c0bad8b01f6041c4ba75f16fa20a2fed P 20b470bb7e1a457eba01ba8453cf2a7e: Deleting consensus metadata
I20260812 06:56:37.866556 7386 ts_tablet_manager.cc:1918] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:37.866743 7386 log.cc:1196] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/c0bad8b01f6041c4ba75f16fa20a2fed
I20260812 06:56:37.867372 7386 ts_tablet_manager.cc:1939] T c0bad8b01f6041c4ba75f16fa20a2fed P 37919c2b404d4ac4b9d7e5f2e4ec0174: Deleting consensus metadata
I20260812 06:56:37.879230 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.834030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.26.226.61:43543)
0812 06:56:37.834274 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.834284 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.850923 (+ 16639us) server_negotiation.cc:407] Connection header received
0812 06:56:37.851186 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.851192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.851289 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.851411 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:37.855693 (+ 4282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.857225 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.866462 (+ 9237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.867211 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.867369 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.870721 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:37.870726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:37.870729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:37.870763 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:37.874788 (+ 4025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.875837 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.875844 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.875848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.876002 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.876554 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.876560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.876564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.876951 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:37.877192 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.878644 (+ 1452us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.878883 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 06:56:37.883783 7385 ts_tablet_manager.cc:1918] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:37.883978 7385 log.cc:1196] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/c0bad8b01f6041c4ba75f16fa20a2fed
I20260812 06:56:37.884687 7385 ts_tablet_manager.cc:1939] T c0bad8b01f6041c4ba75f16fa20a2fed P c79032ca1b154df28ff4759562daa6e5: Deleting consensus metadata
I20260812 06:56:37.898324 7311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.851693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.26.226.60:33887)
0812 06:56:37.851917 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.851923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.878449 (+ 26526us) server_negotiation.cc:407] Connection header received
0812 06:56:37.880131 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.880138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.880228 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.880304 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:37.882291 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.883389 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.885289 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:37.885930 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:37.886045 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:37.890514 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:37.890522 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:37.890529 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:37.890582 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:37.894653 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.895607 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.895614 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.895618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.895806 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:37.896399 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:37.896406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:37.896411 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:37.896897 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:37.897127 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.897725 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.897974 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:56:38.002210 5183 catalog_manager.cc:4841] TS 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865): tablet c0bad8b01f6041c4ba75f16fa20a2fed (table test-table [id=105cfa474612422ebfe632fd93c3fcf5]) successfully deleted
I20260812 06:56:38.002745 5181 catalog_manager.cc:4841] TS 37919c2b404d4ac4b9d7e5f2e4ec0174 (127.26.226.1:42641): tablet c0bad8b01f6041c4ba75f16fa20a2fed (table test-table [id=105cfa474612422ebfe632fd93c3fcf5]) successfully deleted
I20260812 06:56:38.002807 5179 catalog_manager.cc:4841] TS c79032ca1b154df28ff4759562daa6e5 (127.26.226.3:46431): tablet c0bad8b01f6041c4ba75f16fa20a2fed (table test-table [id=105cfa474612422ebfe632fd93c3fcf5]) successfully deleted
I20260812 06:56:38.003327 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 5153
I20260812 06:56:38.066308 7311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:38.047721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36676 (local address 127.26.226.60:33887)
0812 06:56:38.048015 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:38.048025 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:38.049147 (+ 1122us) server_negotiation.cc:407] Connection header received
0812 06:56:38.050383 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:38.050391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:38.050502 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:38.050614 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:38.052327 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:38.053219 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:38.055099 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:38.055756 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:38.055864 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:38.059101 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:38.059111 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:38.059115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:38.059150 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:38.062786 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:38.063747 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:38.063754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:38.063758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:38.063908 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:38.064414 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:38.064421 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:38.064424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:38.064796 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:38.064958 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:38.065711 (+ 753us) server_negotiation.cc:299] Negotiation successful
0812 06:56:38.065927 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 06:56:38.067930 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:38.049102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33232 (local address 127.26.226.61:43543)
0812 06:56:38.049389 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:38.049398 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:38.050148 (+ 750us) server_negotiation.cc:407] Connection header received
0812 06:56:38.050481 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:38.050493 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:38.050574 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:38.050666 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:38.053020 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:38.054232 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:38.056170 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:38.056831 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:38.056963 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:38.060603 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:38.060610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:38.060615 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:38.060655 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:38.064479 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:38.065341 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:38.065347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:38.065351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:38.065505 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:38.065906 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:38.065911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:38.065914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:38.066338 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:38.066833 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:38.067379 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 06:56:38.067629 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 06:56:38.805833 7404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:38.804891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.3:38367)
0812 06:56:38.805394 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:38.805542 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":281,"threads_started":1}
W20260812 06:56:38.806399 7285 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:36667 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
I20260812 06:56:38.811990 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:38.811191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.2:48023)
0812 06:56:38.811651 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:38.811826 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":222,"threads_started":1}
W20260812 06:56:38.812500 7045 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:36667 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
I20260812 06:56:38.816792 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:38.815844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.1:36293)
0812 06:56:38.816403 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:38.816557 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":296,"threads_started":1}
W20260812 06:56:38.817272 6802 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:36667 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
I20260812 06:56:39.330917 7409 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f4a6c926fc984692809b846da0948692)
I20260812 06:56:39.331274 7409 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } }
I20260812 06:56:39.332747 7409 leader_election.cc:288] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4a6c926fc984692809b846da0948692 (127.26.226.62:36667), 10df32c12dff4c2e91adacad56bc3efb (127.26.226.61:43543)
I20260812 06:56:39.334077 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.332952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:39154)
0812 06:56:39.333559 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.333741 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":270,"threads_started":1}
I20260812 06:56:39.335425 7411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f4a6c926fc984692809b846da0948692)
W20260812 06:56:39.335683 6481 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
I20260812 06:56:39.335739 7411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } }
I20260812 06:56:39.337473 7411 leader_election.cc:288] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4a6c926fc984692809b846da0948692 (127.26.226.62:36667), e2e89365619e4408b2ad53aee78acff8 (127.26.226.60:33887)
I20260812 06:56:39.337713 7412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.337106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:39158)
0812 06:56:39.337288 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.337479 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 06:56:39.338810 7420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.337505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:39172)
0812 06:56:39.338382 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.338532 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":791,"threads_started":1}
W20260812 06:56:39.338914 6481 leader_election.cc:334] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f4a6c926fc984692809b846da0948692 (127.26.226.62:36667): Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
W20260812 06:56:39.339884 6159 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
I20260812 06:56:39.342087 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.341405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:39186)
0812 06:56:39.341617 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.341858 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 06:56:39.342728 6159 leader_election.cc:334] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f4a6c926fc984692809b846da0948692 (127.26.226.62:36667): Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
I20260812 06:56:39.343050 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.333709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:43543 (local address 127.0.0.1:33244)
0812 06:56:39.334343 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.334366 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:39.334585 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:39.335698 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:39.335704 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:39.335764 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:39.335994 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.336007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.337645 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.337653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.340190 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.340207 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.342084 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.342090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.342399 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.342425 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 06:56:39.342558 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 06:56:39.342717 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":105,"thread_start_us":1643,"threads_started":1}
I20260812 06:56:39.343247 7417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.333886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33244 (local address 127.26.226.61:43543)
0812 06:56:39.335312 (+ 1426us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.335322 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.335346 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:56:39.335431 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.335435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.335546 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.335660 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:39.336204 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.337426 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.340391 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.341895 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.342056 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.342158 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.342584 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.342720 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":1251,"thread_start_us":960,"threads_started":1}
I20260812 06:56:39.343827 6197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e2e89365619e4408b2ad53aee78acff8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "10df32c12dff4c2e91adacad56bc3efb" is_pre_election: true
I20260812 06:56:39.344230 6197 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2e89365619e4408b2ad53aee78acff8 in term 1.
I20260812 06:56:39.344794 6481 leader_election.cc:302] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10df32c12dff4c2e91adacad56bc3efb, e2e89365619e4408b2ad53aee78acff8; no voters: f4a6c926fc984692809b846da0948692
I20260812 06:56:39.345184 7409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:56:39.345340 7409 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f4a6c926fc984692809b846da0948692)
I20260812 06:56:39.345453 7409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:56:39.347527 7420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.337897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:33887 (local address 127.0.0.1:36680)
0812 06:56:39.339138 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.339157 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:39.339296 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:39.340900 (+ 1604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:39.340905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:39.340935 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:39.341374 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.341389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.342809 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.342817 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.344991 (+ 2174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.345007 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.346766 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.346773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.346965 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.347000 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 06:56:39.347073 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 06:56:39.347203 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1090,"mutex_wait_us":3,"thread_start_us":266,"threads_started":1}
I20260812 06:56:39.348394 7424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.339241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.26.226.60:33887)
0812 06:56:39.340493 (+ 1252us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.340501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.340526 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:56:39.340585 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.340590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.340661 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.340896 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:39.341587 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.342679 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.345145 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.346533 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.347763 (+ 1230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.347878 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.347928 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.348048 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1135,"thread_start_us":997,"threads_started":1}
I20260812 06:56:39.348897 7409 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } }
I20260812 06:56:39.349046 6530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "10df32c12dff4c2e91adacad56bc3efb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e2e89365619e4408b2ad53aee78acff8" is_pre_election: true
I20260812 06:56:39.349411 6530 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10df32c12dff4c2e91adacad56bc3efb in current term 2: Already voted for candidate e2e89365619e4408b2ad53aee78acff8 in this term.
I20260812 06:56:39.349670 7409 leader_election.cc:288] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [CANDIDATE]: Term 2 election: Requested vote from peers f4a6c926fc984692809b846da0948692 (127.26.226.62:36667), 10df32c12dff4c2e91adacad56bc3efb (127.26.226.61:43543)
I20260812 06:56:39.350198 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.349734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:39192)
0812 06:56:39.349955 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.350047 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 06:56:39.350301 6161 leader_election.cc:302] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10df32c12dff4c2e91adacad56bc3efb; no voters: e2e89365619e4408b2ad53aee78acff8, f4a6c926fc984692809b846da0948692
I20260812 06:56:39.350311 6197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e2e89365619e4408b2ad53aee78acff8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "10df32c12dff4c2e91adacad56bc3efb"
I20260812 06:56:39.350592 6197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:56:39.351357 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.350965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:39208)
0812 06:56:39.351144 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.351238 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 06:56:39.351531 6481 leader_election.cc:334] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f4a6c926fc984692809b846da0948692 (127.26.226.62:36667): Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
I20260812 06:56:39.353008 6197 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2e89365619e4408b2ad53aee78acff8 in term 2.
I20260812 06:56:39.353189 7411 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:56:39.353468 6481 leader_election.cc:302] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10df32c12dff4c2e91adacad56bc3efb, e2e89365619e4408b2ad53aee78acff8; no voters: f4a6c926fc984692809b846da0948692
I20260812 06:56:39.353799 7409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:56:39.354264 7409 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [term 2 LEADER]: Becoming Leader. State: Replica: e2e89365619e4408b2ad53aee78acff8, State: Running, Role: LEADER
I20260812 06:56:39.354635 7409 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } }
I20260812 06:56:39.357113 7428 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2e89365619e4408b2ad53aee78acff8. Latest consensus state: current_term: 2 leader_uuid: "e2e89365619e4408b2ad53aee78acff8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:39.357499 7428 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:39.358268 7430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:39.362013 7430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:39.362932 7430 catalog_manager.cc:1259] Loaded cluster ID: 993356870845481c94b7918d6e35d6eb
I20260812 06:56:39.363006 7430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:39.364326 7430 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:39.365218 7430 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Loaded TSK: 0
I20260812 06:56:39.366381 7430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:39.415222 6513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:39.422719 6513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:39.427343 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.426850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:46862)
0812 06:56:39.427064 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.427178 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 06:56:39.427875 6197 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [term 2 FOLLOWER]: Refusing update from remote peer e2e89365619e4408b2ad53aee78acff8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 06:56:39.428524 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.428126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:46866)
0812 06:56:39.428291 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.428397 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 06:56:39.428768 6481 consensus_peers.cc:487] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 -> Peer f4a6c926fc984692809b846da0948692 (127.26.226.62:36667): Couldn't send request to peer f4a6c926fc984692809b846da0948692. Status: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 06:56:39.428819 7409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 06:56:39.431866 7411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2e89365619e4408b2ad53aee78acff8. Latest consensus state: current_term: 2 leader_uuid: "e2e89365619e4408b2ad53aee78acff8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:39.432181 7411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:56:39.433799 7409 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "e2e89365619e4408b2ad53aee78acff8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:39.434195 7409 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:39.434574 7411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "e2e89365619e4408b2ad53aee78acff8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f4a6c926fc984692809b846da0948692" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36667 } } peers { permanent_uuid: "10df32c12dff4c2e91adacad56bc3efb" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 43543 } attrs { promote: false } } peers { permanent_uuid: "e2e89365619e4408b2ad53aee78acff8" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 33887 } attrs { promote: false } } }
I20260812 06:56:39.434859 7411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10df32c12dff4c2e91adacad56bc3efb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:56:39.461750 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.443223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:42641 (local address 127.0.0.1:50442)
0812 06:56:39.443516 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.443557 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:39.443674 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:39.455147 (+ 11473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:39.455152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:39.455182 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:39.455377 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.455385 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.457008 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.457020 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.459302 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.459319 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.460889 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.460895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.461099 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.461125 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 06:56:39.461235 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 06:56:39.461380 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":64}
I20260812 06:56:39.461745 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.443389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50442 (local address 127.26.226.1:42641)
0812 06:56:39.454571 (+ 11182us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.454581 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.454609 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:56:39.454709 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.454717 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.454825 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.455504 (+ 679us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:39.455634 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.456835 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.459829 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.460725 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.460879 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.460994 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.461275 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.461419 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10999,"thread_start_us":11561,"threads_started":1}
I20260812 06:56:39.462241 7412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.443766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:46431 (local address 127.0.0.1:40402)
0812 06:56:39.443914 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.443931 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:39.444048 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:39.455836 (+ 11788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:39.455841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:39.455865 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:39.456153 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.456174 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.457574 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.457579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.459730 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.459744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.461439 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.461446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.461620 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.461660 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 06:56:39.461730 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:56:39.461876 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":25}
I20260812 06:56:39.462986 6687 tablet_service.cc:1467] Processing CreateTablet for tablet e8359757175e4b339268ad0cd0214779 (DEFAULT_TABLE table=test-table [id=25eb203477094f1fbf7495bf7d6ef890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:39.463166 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.444311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40402 (local address 127.26.226.3:46431)
0812 06:56:39.455407 (+ 11096us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.455417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.455443 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:56:39.455534 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.455542 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.455658 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.455796 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:39.456457 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.457422 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.459915 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.461176 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.462501 (+ 1325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.462626 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.462695 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.462850 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10924,"thread_start_us":10780,"threads_started":1}
I20260812 06:56:39.463495 6687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8359757175e4b339268ad0cd0214779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:39.463840 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.444435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33865 (local address 127.0.0.1:48042)
0812 06:56:39.454414 (+ 9979us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.454443 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:39.454608 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:39.456703 (+ 2095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:39.456708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:39.456732 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:39.457216 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.457233 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.458559 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.458569 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.460838 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.460856 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.463035 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.463041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.463224 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.463262 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:56:39.463358 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 06:56:39.463484 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9797,"mutex_wait_us":6,"thread_start_us":10262,"threads_started":1}
I20260812 06:56:39.464668 7171 tablet_service.cc:1467] Processing CreateTablet for tablet e8359757175e4b339268ad0cd0214779 (DEFAULT_TABLE table=test-table [id=25eb203477094f1fbf7495bf7d6ef890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:39.464828 7442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.444974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48042 (local address 127.26.226.2:33865)
0812 06:56:39.456405 (+ 11431us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.456410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.456427 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:56:39.456487 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.456491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.456556 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.456692 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:39.457437 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.458181 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.461178 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.462649 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.462790 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.462884 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.464370 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.464530 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11271,"thread_start_us":11165,"threads_started":1}
I20260812 06:56:39.465171 7171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8359757175e4b339268ad0cd0214779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:39.468847 6939 tablet_service.cc:1467] Processing CreateTablet for tablet e8359757175e4b339268ad0cd0214779 (DEFAULT_TABLE table=test-table [id=25eb203477094f1fbf7495bf7d6ef890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:39.469327 6939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8359757175e4b339268ad0cd0214779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:39.469511 7443 tablet_bootstrap.cc:492] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174: Bootstrap starting.
I20260812 06:56:39.470548 7444 tablet_bootstrap.cc:492] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5: Bootstrap starting.
I20260812 06:56:39.471602 7443 tablet_bootstrap.cc:654] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:39.472430 7444 tablet_bootstrap.cc:654] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:39.474855 7444 tablet_bootstrap.cc:492] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5: No bootstrap required, opened a new log
I20260812 06:56:39.474998 7449 tablet_bootstrap.cc:492] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e: Bootstrap starting.
I20260812 06:56:39.475011 7444 ts_tablet_manager.cc:1397] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 06:56:39.475857 7444 raft_consensus.cc:348] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:39.476156 7444 raft_consensus.cc:374] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:39.476271 7444 raft_consensus.cc:729] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79032ca1b154df28ff4759562daa6e5, State: Initialized, Role: FOLLOWER
I20260812 06:56:39.476514 7444 consensus_queue.cc:260] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:39.476823 7449 tablet_bootstrap.cc:654] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:39.476929 7443 tablet_bootstrap.cc:492] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174: No bootstrap required, opened a new log
I20260812 06:56:39.477104 7443 ts_tablet_manager.cc:1397] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260812 06:56:39.477744 7443 raft_consensus.cc:348] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:39.478006 7443 raft_consensus.cc:374] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:39.478128 7443 raft_consensus.cc:729] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37919c2b404d4ac4b9d7e5f2e4ec0174, State: Initialized, Role: FOLLOWER
I20260812 06:56:39.478411 7443 consensus_queue.cc:260] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:39.479341 7443 ts_tablet_manager.cc:1428] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:39.480861 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.479673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.1:42803)
0812 06:56:39.480394 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.480559 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":316,"threads_started":1}
I20260812 06:56:39.482033 7459 raft_consensus.cc:482] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:39.482627 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.482269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.1:57845)
0812 06:56:39.482402 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.482486 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 06:56:39.482264 7459 raft_consensus.cc:504] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:39.484161 6804 heartbeater.cc:503] Master 127.26.226.60:33887 was elected leader, sending a full tablet report...
I20260812 06:56:39.484227 7449 tablet_bootstrap.cc:492] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e: No bootstrap required, opened a new log
I20260812 06:56:39.484385 7449 ts_tablet_manager.cc:1397] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20260812 06:56:39.485328 7449 raft_consensus.cc:348] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:39.485577 7449 raft_consensus.cc:374] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:39.485678 7449 raft_consensus.cc:729] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b470bb7e1a457eba01ba8453cf2a7e, State: Initialized, Role: FOLLOWER
I20260812 06:56:39.485962 7449 consensus_queue.cc:260] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:39.488363 7444 ts_tablet_manager.cc:1428] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5: Time spent starting tablet: real 0.013s user 0.003s sys 0.008s
I20260812 06:56:39.490882 7287 heartbeater.cc:503] Master 127.26.226.60:33887 was elected leader, sending a full tablet report...
I20260812 06:56:39.493149 7449 ts_tablet_manager.cc:1428] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e: Time spent starting tablet: real 0.009s user 0.002s sys 0.006s
I20260812 06:56:39.494619 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.488905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.3:56425)
0812 06:56:39.493974 (+ 5069us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.494255 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1483,"thread_start_us":1583,"threads_started":1}
I20260812 06:56:39.497037 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.485428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:51055 (local address 127.26.226.3:46431)
0812 06:56:39.485604 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.485611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.486034 (+ 423us) server_negotiation.cc:407] Connection header received
0812 06:56:39.486153 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.486158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.486246 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.486356 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:39.487206 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.488388 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.493689 (+ 5301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.495381 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.495559 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.495670 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.496528 (+ 858us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.496710 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:56:39.498333 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.485044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:46431 (local address 127.26.226.1:51055)
0812 06:56:39.485744 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.485771 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:39.485912 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:39.486567 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:39.486575 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:39.486611 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:39.486995 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.487010 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.488517 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.488526 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.491316 (+ 2790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.491338 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.495917 (+ 4579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.495928 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.496238 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.496299 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 06:56:39.497880 (+ 1581us) client_negotiation.cc:240] Negotiation successful
0812 06:56:39.498046 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591}
I20260812 06:56:39.505155 7459 leader_election.cc:288] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c79032ca1b154df28ff4759562daa6e5 (127.26.226.3:46431), 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865)
I20260812 06:56:39.509443 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.508290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.3:56145)
0812 06:56:39.508911 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.509137 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":310,"threads_started":1}
I20260812 06:56:39.509608 7208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8359757175e4b339268ad0cd0214779" candidate_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79032ca1b154df28ff4759562daa6e5" is_pre_election: true
I20260812 06:56:39.510090 7208 raft_consensus.cc:2454] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37919c2b404d4ac4b9d7e5f2e4ec0174 in term 0.
I20260812 06:56:39.510267 7442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.488108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:49367 (local address 127.26.226.2:33865)
0812 06:56:39.488356 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.488365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.499368 (+ 11003us) server_negotiation.cc:407] Connection header received
0812 06:56:39.500082 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.500088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.500165 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.500239 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:39.501145 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.502231 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.506733 (+ 4502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.508307 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.508481 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.508600 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.509715 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.509926 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:56:39.510295 7048 heartbeater.cc:503] Master 127.26.226.60:33887 was elected leader, sending a full tablet report...
I20260812 06:56:39.510910 6618 leader_election.cc:302] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37919c2b404d4ac4b9d7e5f2e4ec0174, c79032ca1b154df28ff4759562daa6e5; no voters:
I20260812 06:56:39.515425 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.493828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.2:51397)
0812 06:56:39.515082 (+ 21254us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.515233 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21139,"threads_started":1}
I20260812 06:56:39.515697 7459 raft_consensus.cc:2781] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:39.515884 7459 raft_consensus.cc:482] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:39.516110 7459 raft_consensus.cc:3037] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:39.515959 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.487659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33865 (local address 127.26.226.1:49367)
0812 06:56:39.499272 (+ 11613us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.499293 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:39.499695 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:39.500446 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:39.500451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:39.500476 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:39.500649 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.500658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.502435 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.502440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.504251 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:39.504268 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.508790 (+ 4522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.508795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:39.508988 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.509045 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 06:56:39.510486 (+ 1441us) client_negotiation.cc:240] Negotiation successful
0812 06:56:39.510714 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11499,"thread_start_us":26276,"threads_started":1}
I20260812 06:56:39.517094 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.516659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.2:42815)
0812 06:56:39.516846 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.516936 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 06:56:39.517656 6967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8359757175e4b339268ad0cd0214779" candidate_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" is_pre_election: true
I20260812 06:56:39.517907 6967 raft_consensus.cc:2454] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37919c2b404d4ac4b9d7e5f2e4ec0174 in term 0.
I20260812 06:56:39.519389 7459 raft_consensus.cc:504] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:39.520231 7459 leader_election.cc:288] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [CANDIDATE]: Term 1 election: Requested vote from peers c79032ca1b154df28ff4759562daa6e5 (127.26.226.3:46431), 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865)
I20260812 06:56:39.520603 7208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8359757175e4b339268ad0cd0214779" candidate_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79032ca1b154df28ff4759562daa6e5"
I20260812 06:56:39.520695 6967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8359757175e4b339268ad0cd0214779" candidate_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20b470bb7e1a457eba01ba8453cf2a7e"
I20260812 06:56:39.520815 7208 raft_consensus.cc:3037] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:39.520947 6967 raft_consensus.cc:3037] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:39.523408 6967 raft_consensus.cc:2454] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37919c2b404d4ac4b9d7e5f2e4ec0174 in term 1.
I20260812 06:56:39.523428 7208 raft_consensus.cc:2454] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37919c2b404d4ac4b9d7e5f2e4ec0174 in term 1.
I20260812 06:56:39.523844 6623 leader_election.cc:302] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20b470bb7e1a457eba01ba8453cf2a7e, 37919c2b404d4ac4b9d7e5f2e4ec0174; no voters:
I20260812 06:56:39.524231 7459 raft_consensus.cc:2781] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:39.524389 7459 raft_consensus.cc:686] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 1 LEADER]: Becoming Leader. State: Replica: 37919c2b404d4ac4b9d7e5f2e4ec0174, State: Running, Role: LEADER
I20260812 06:56:39.524818 7459 consensus_queue.cc:237] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } } peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } }
I20260812 06:56:39.525352 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.524893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.1:60803)
0812 06:56:39.525034 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.525177 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 06:56:39.528765 6512 catalog_manager.cc:5495] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37919c2b404d4ac4b9d7e5f2e4ec0174 (127.26.226.1). New cstate: current_term: 1 leader_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37919c2b404d4ac4b9d7e5f2e4ec0174" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 42641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 } health_report { overall_health: UNKNOWN } } }
I20260812 06:56:39.554037 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.548706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50456 (local address 127.26.226.1:42641)
0812 06:56:39.548933 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.548940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.548962 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:56:39.549221 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.549226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.549308 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.549467 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:39.549848 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.550534 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.551843 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.552338 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.552474 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.552894 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:39.553440 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:39.553609 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.553666 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.553767 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:56:39.558192 6967 raft_consensus.cc:1261] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e [term 1 FOLLOWER]: Refusing update from remote peer 37919c2b404d4ac4b9d7e5f2e4ec0174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:39.558553 7208 raft_consensus.cc:1261] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5 [term 1 FOLLOWER]: Refusing update from remote peer 37919c2b404d4ac4b9d7e5f2e4ec0174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:39.559002 7455 consensus_queue.cc:1035] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20b470bb7e1a457eba01ba8453cf2a7e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:39.559480 7459 consensus_queue.cc:1035] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79032ca1b154df28ff4759562daa6e5" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:39.562336 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.561395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.3:53771)
0812 06:56:39.562062 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.562163 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552}
I20260812 06:56:39.562697 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.561810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.2:55373)
0812 06:56:39.562296 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.562435 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307}
I20260812 06:56:39.568588 7478 mvcc.cc:205] Tried to move back new op lower bound from 7317576906983186432 to 7317576906858156032. Current Snapshot: MvccSnapshot[applied={T|T < 7317576906983186432}]
I20260812 06:56:39.569355 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.568820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.3:44973)
0812 06:56:39.569059 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.569171 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260812 06:56:39.572227 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.568328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.1:34423)
0812 06:56:39.571922 (+ 3594us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.572023 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3485}
I20260812 06:56:39.577684 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.577171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.1:46425)
0812 06:56:39.577393 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.577496 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260812 06:56:39.585275 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.584800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.26.226.2:51059)
0812 06:56:39.584975 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.585100 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 06:56:39.590260 6512 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:39.590543 6512 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:39.597254 6512 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2e89365619e4408b2ad53aee78acff8: Sending DeleteTablet for 3 replicas of tablet e8359757175e4b339268ad0cd0214779
I20260812 06:56:39.598286 6939 tablet_service.cc:1514] Processing DeleteTablet for tablet e8359757175e4b339268ad0cd0214779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:39 UTC) from {username='oryx', principal='oryx/127.26.226.60@KRBTEST.COM'} at 127.0.0.1:48042
I20260812 06:56:39.598307 6687 tablet_service.cc:1514] Processing DeleteTablet for tablet e8359757175e4b339268ad0cd0214779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:39 UTC) from {username='oryx', principal='oryx/127.26.226.60@KRBTEST.COM'} at 127.0.0.1:50442
I20260812 06:56:39.598369 7171 tablet_service.cc:1514] Processing DeleteTablet for tablet e8359757175e4b339268ad0cd0214779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:39 UTC) from {username='oryx', principal='oryx/127.26.226.60@KRBTEST.COM'} at 127.0.0.1:40402
I20260812 06:56:39.599100 7497 tablet_replica.cc:330] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e: stopping tablet replica
I20260812 06:56:39.599720 7496 tablet_replica.cc:330] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174: stopping tablet replica
I20260812 06:56:39.600093 7496 raft_consensus.cc:2229] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:39.600739 6939 tablet_service.cc:1514] Processing DeleteTablet for tablet e8359757175e4b339268ad0cd0214779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:39 UTC) from {username='oryx', principal='oryx/127.26.226.60@KRBTEST.COM'} at 127.0.0.1:48042
I20260812 06:56:39.600901 7497 raft_consensus.cc:2229] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:39.601205 7497 raft_consensus.cc:2258] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:39.600801 7496 raft_consensus.cc:2258] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:39.601938 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 6462
I20260812 06:56:39.602375 7497 ts_tablet_manager.cc:1905] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:39.609066 7497 ts_tablet_manager.cc:1918] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:39.609205 7497 log.cc:1196] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/e8359757175e4b339268ad0cd0214779
I20260812 06:56:39.609669 7497 ts_tablet_manager.cc:1939] T e8359757175e4b339268ad0cd0214779 P 20b470bb7e1a457eba01ba8453cf2a7e: Deleting consensus metadata
I20260812 06:56:39.609678 7496 ts_tablet_manager.cc:1905] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:39.610828 6490 catalog_manager.cc:4841] TS 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865): tablet e8359757175e4b339268ad0cd0214779 (table test-table [id=25eb203477094f1fbf7495bf7d6ef890]) successfully deleted
W20260812 06:56:39.611806 6490 catalog_manager.cc:4808] TS 20b470bb7e1a457eba01ba8453cf2a7e (127.26.226.2:33865): delete failed for tablet e8359757175e4b339268ad0cd0214779 because the tablet was not found. No further retry: Not found: Tablet not found: e8359757175e4b339268ad0cd0214779
I20260812 06:56:39.613632 7498 tablet_replica.cc:330] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5: stopping tablet replica
I20260812 06:56:39.613912 7498 raft_consensus.cc:2229] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:39.614323 7498 raft_consensus.cc:2258] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:39.615520 7498 ts_tablet_manager.cc:1905] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:39.621078 7496 ts_tablet_manager.cc:1918] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:39.621299 7496 log.cc:1196] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/e8359757175e4b339268ad0cd0214779
I20260812 06:56:39.622000 7496 ts_tablet_manager.cc:1939] T e8359757175e4b339268ad0cd0214779 P 37919c2b404d4ac4b9d7e5f2e4ec0174: Deleting consensus metadata
I20260812 06:56:39.622191 7498 ts_tablet_manager.cc:1918] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:39.622354 7498 log.cc:1196] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/e8359757175e4b339268ad0cd0214779
I20260812 06:56:39.622803 7498 ts_tablet_manager.cc:1939] T e8359757175e4b339268ad0cd0214779 P c79032ca1b154df28ff4759562daa6e5: Deleting consensus metadata
I20260812 06:56:39.623514 6480 catalog_manager.cc:4841] TS 37919c2b404d4ac4b9d7e5f2e4ec0174 (127.26.226.1:42641): tablet e8359757175e4b339268ad0cd0214779 (table test-table [id=25eb203477094f1fbf7495bf7d6ef890]) successfully deleted
I20260812 06:56:39.623942 6478 catalog_manager.cc:4841] TS c79032ca1b154df28ff4759562daa6e5 (127.26.226.3:46431): tablet e8359757175e4b339268ad0cd0214779 (table test-table [id=25eb203477094f1fbf7495bf7d6ef890]) successfully deleted
I20260812 06:56:39.631791 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.603093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.26.226.1:42641)
0812 06:56:39.603399 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.603405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.615425 (+ 12020us) server_negotiation.cc:407] Connection header received
0812 06:56:39.615493 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.615498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.615607 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.615711 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:39.617859 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.619191 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.621199 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.621907 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.622021 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.625676 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:39.625690 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:39.625713 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:39.625748 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:39.628888 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:39.629685 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:39.629692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:39.629694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:39.629867 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:39.630280 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:39.630284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:39.630285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:39.630630 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:39.630959 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.631277 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.631434 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260812 06:56:39.632570 6794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:39.734702 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 6462
I20260812 06:56:39.770292 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 6822
I20260812 06:56:39.785655 7442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.771301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48058 (local address 127.26.226.2:33865)
0812 06:56:39.771505 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.771512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.771606 (+ 94us) server_negotiation.cc:407] Connection header received
0812 06:56:39.771837 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.771841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.771907 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.772012 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:39.773409 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.774161 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.775374 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.775792 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.775930 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.778674 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:39.778691 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:39.778716 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:39.778755 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:39.781790 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:39.782812 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:39.782823 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:39.782825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:39.782969 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:39.783364 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:39.783369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:39.783372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:39.784027 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:39.784642 (+ 615us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.785009 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.785311 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:56:39.786365 7037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:39.885921 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 6822
I20260812 06:56:39.886935 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.886349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:46868)
0812 06:56:39.886626 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.886741 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260812 06:56:39.888377 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.887879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36667 (local address 127.0.0.1:46878)
0812 06:56:39.888147 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:39.888240 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:36667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260812 06:56:39.911259 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 7065
I20260812 06:56:39.924940 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.912176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.26.226.3:46431)
0812 06:56:39.912352 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.912357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.912377 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:56:39.912459 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.912462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.912537 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.912619 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:39.914058 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.914827 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.916048 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.916621 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.916725 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.919214 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:39.919230 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:39.919247 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:39.919280 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:39.922308 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:39.922970 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:39.922978 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:39.922980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:39.923159 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:39.923480 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:39.923484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:39.923485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:39.923852 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:39.924172 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.924490 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.924664 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:56:39.925724 7276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:40.054296 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 7065
I20260812 06:56:40.088858 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 5950
I20260812 06:56:40.089951 6230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:40.209106 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 5950
I20260812 06:56:40.240676 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 6117
I20260812 06:56:40.241767 6556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:40.366343 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 6117
2026-08-12T06:56:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10075 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:40 dist-test-slave-t3q3 krb5kdc[7515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:40 dist-test-slave-t3q3 krb5kdc[7515](info): set up 2 sockets
Aug 12 06:56:40 dist-test-slave-t3q3 krb5kdc[7515](info): commencing operation
krb5kdc: starting...
W20260812 06:56:41.732534 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:56:41 dist-test-slave-t3q3 krb5kdc[7515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.62.keytab.
Entry for principal oryx/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.62.keytab.
I20260812 06:56:42.016396 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:36507
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.62.keytab
--principal=oryx/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:36507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:42.365229 7534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:42.365499 7534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:42.365589 7534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:42.372665 7534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:42.372772 7534 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 06:56:42.372814 7534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:42.372874 7534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:42.372923 7534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:42.373004 7534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:42.382697 7534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:36507
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.62.keytab
--principal=oryx/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:36507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:42.384027 7534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:42.385389 7534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:42.398088 7540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:42.398144 7544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:42.398167 7541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:42.399101 7534 server_base.cc:1034] running on GCE node
I20260812 06:56:42.399966 7534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:42.401500 7534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:42.402761 7534 hybrid_clock.cc:648] HybridClock initialized: now 1786517802402698 us; error 41 us; skew 500 ppm
I20260812 06:56:42.403148 7534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:42 dist-test-slave-t3q3 krb5kdc[7515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517802, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:42.406409 7534 init.cc:377] Logged in from keytab as oryx/127.26.226.62@KRBTEST.COM (short username oryx)
I20260812 06:56:42.407862 7534 webserver.cc:460] Webserver started at http://127.26.226.62:38587/ using document root <none> and password file <none>
I20260812 06:56:42.408617 7534 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:42.408756 7534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:42.409106 7534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:42.412238 7534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837"
format_stamp: "Formatted at 2026-08-12 06:56:42 on dist-test-slave-t3q3"
server_key: "0dce328831409471654d38e7d210f4f0"
server_key_iv: "b8833752f45b2ad67c38ac506524d94a"
server_key_version: "encryptionkey@0"
I20260812 06:56:42.413062 7534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837"
format_stamp: "Formatted at 2026-08-12 06:56:42 on dist-test-slave-t3q3"
server_key: "0dce328831409471654d38e7d210f4f0"
server_key_iv: "b8833752f45b2ad67c38ac506524d94a"
server_key_version: "encryptionkey@0"
I20260812 06:56:42.418326 7534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:56:42.421911 7551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:42.423157 7534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:42.423333 7534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837"
format_stamp: "Formatted at 2026-08-12 06:56:42 on dist-test-slave-t3q3"
server_key: "0dce328831409471654d38e7d210f4f0"
server_key_iv: "b8833752f45b2ad67c38ac506524d94a"
server_key_version: "encryptionkey@0"
I20260812 06:56:42.423563 7534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:42.445875 7534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:42.450165 7534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:42.450452 7534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:42.473101 7534 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:36507
I20260812 06:56:42.473109 7663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:36507 every 8 connection(s)
I20260812 06:56:42.474999 7534 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:42.475550 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 7534
I20260812 06:56:42.475777 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:42.476648 27528 external_mini_cluster.cc:1428] Setting key 27e418a21b6abe5b4f6712cdf83adeda
I20260812 06:56:42.479487 7664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:42.489831 7664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837: Bootstrap starting.
Aug 12 06:56:42 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:42.493003 7664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:42.494076 7664 log.cc:824] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:42.497316 7664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837: No bootstrap required, opened a new log
I20260812 06:56:42.498705 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:42.478718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34854 (local address 127.26.226.62:36507)
0812 06:56:42.479515 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:42.479528 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:42.479567 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:56:42.481341 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:42.481362 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:42.481737 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:42.482151 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:42.483634 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:42.485110 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:42.486360 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:42.487064 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:42.487309 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:42.491405 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:42.491413 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:42.491430 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:42.491466 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:42.494699 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:42.495421 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:42.495427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:42.495433 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:42.495600 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:42.496032 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:42.496035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:42.496037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:42.496585 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:42.496825 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:42.497296 (+ 471us) server_negotiation.cc:299] Negotiation successful
0812 06:56:42.497755 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":369,"threads_started":1}
I20260812 06:56:42.505457 7664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36507 } }
I20260812 06:56:42.505810 7664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:42.505908 7664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cf7aedca9cd4ac4a3696cbf3bdd0837, State: Initialized, Role: FOLLOWER
I20260812 06:56:42.506455 7664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36507 } }
I20260812 06:56:42.506696 7664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:42.506857 7664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:42.507038 7664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:42.509435 7664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36507 } }
I20260812 06:56:42.509936 7664 leader_election.cc:302] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cf7aedca9cd4ac4a3696cbf3bdd0837; no voters:
I20260812 06:56:42.510550 7664 leader_election.cc:288] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:42.510705 7680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:42.511595 7680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [term 1 LEADER]: Becoming Leader. State: Replica: 4cf7aedca9cd4ac4a3696cbf3bdd0837, State: Running, Role: LEADER
I20260812 06:56:42.512176 7680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36507 } }
I20260812 06:56:42.512948 7664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:42.515437 7683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cf7aedca9cd4ac4a3696cbf3bdd0837. Latest consensus state: current_term: 1 leader_uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36507 } } }
I20260812 06:56:42.515808 7683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:42.515579 7681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cf7aedca9cd4ac4a3696cbf3bdd0837" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36507 } } }
I20260812 06:56:42.516042 7681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:42.522286 7689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:42.528002 7689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:42.537815 7689 catalog_manager.cc:1347] Generated new cluster ID: ed5b74d8724e4510a52e4ef73274dee4
I20260812 06:56:42.537966 7689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:42.550475 7689 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:42.551874 7689 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:42.560385 7689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837: Generated new TSK 0
I20260812 06:56:42.561528 7689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab.
Entry for principal oryx/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab.
I20260812 06:56:42.635618 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36507
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab
--principal=oryx/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:42.985133 7710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:42.985471 7710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:42.985670 7710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:42.993135 7710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:42.993297 7710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:42.993441 7710 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:56:42.993580 7710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:43.003594 7710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab
--principal=oryx/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.226.62:36507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:43.005017 7710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:43.006404 7710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:43.020509 7719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:43.020555 7717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:43.020789 7716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:43.021515 7710 server_base.cc:1034] running on GCE node
I20260812 06:56:43.022225 7710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:43.023140 7710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:43.024387 7710 hybrid_clock.cc:648] HybridClock initialized: now 1786517803024324 us; error 47 us; skew 500 ppm
I20260812 06:56:43.024706 7710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:43 dist-test-slave-t3q3 krb5kdc[7515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517803, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:43.027990 7710 init.cc:377] Logged in from keytab as oryx/127.26.226.1@KRBTEST.COM (short username oryx)
I20260812 06:56:43.029538 7710 webserver.cc:460] Webserver started at http://127.26.226.1:38051/ using document root <none> and password file <none>
I20260812 06:56:43.030243 7710 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:43.030337 7710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:43.030723 7710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:43.033737 7710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "7436a98362ff4489a0255322b0a0753e"
format_stamp: "Formatted at 2026-08-12 06:56:43 on dist-test-slave-t3q3"
server_key: "21fc4d68f9b04a19b427fb59b3df96ea"
server_key_iv: "29020f13377dabd2d266f6fdb1377f76"
server_key_version: "encryptionkey@0"
I20260812 06:56:43.034543 7710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "7436a98362ff4489a0255322b0a0753e"
format_stamp: "Formatted at 2026-08-12 06:56:43 on dist-test-slave-t3q3"
server_key: "21fc4d68f9b04a19b427fb59b3df96ea"
server_key_iv: "29020f13377dabd2d266f6fdb1377f76"
server_key_version: "encryptionkey@0"
I20260812 06:56:43.039773 7710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260812 06:56:43.043215 7735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:43.044634 7710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:43.044780 7710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "7436a98362ff4489a0255322b0a0753e"
format_stamp: "Formatted at 2026-08-12 06:56:43 on dist-test-slave-t3q3"
server_key: "21fc4d68f9b04a19b427fb59b3df96ea"
server_key_iv: "29020f13377dabd2d266f6fdb1377f76"
server_key_version: "encryptionkey@0"
I20260812 06:56:43.044971 7710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:43.068415 7710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:43.072886 7710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:43.073143 7710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:43.075733 7710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:43.075821 7710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:43.076020 7710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:43.076123 7710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 12 06:56:43 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517803, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:43.099749 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:43.076390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:45869 (local address 127.26.226.62:36507)
0812 06:56:43.077089 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:43.077096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:43.078168 (+ 1072us) server_negotiation.cc:407] Connection header received
0812 06:56:43.080552 (+ 2384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:43.080558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:43.080648 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:43.080747 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:43.083190 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.084266 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.086717 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.087354 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.087520 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:43.093038 (+ 5518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:43.093046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:43.093054 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:43.093100 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:43.096624 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:43.097351 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:43.097360 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:43.097364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:43.097491 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:43.097959 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:43.097964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:43.097969 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:43.098246 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:43.098407 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:43.099155 (+ 748us) server_negotiation.cc:299] Negotiation successful
0812 06:56:43.099377 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":455,"threads_started":1}
I20260812 06:56:43.101202 7755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:43.077098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36507 (local address 127.26.226.1:45869)
0812 06:56:43.077969 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:43.078034 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:43.080182 (+ 2148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:43.081317 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:43.081325 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:43.082126 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:43.082982 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:43.083013 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.084432 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.084438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:43.086468 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:43.086480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.087685 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.087694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:43.087937 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:43.088867 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:43.088911 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:43.092735 (+ 3824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:43.096772 (+ 4037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:43.096782 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:43.096797 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:43.097222 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:43.097612 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:43.097617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:43.097621 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:43.097830 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:43.098380 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:43.098388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:43.098748 (+ 360us) client_negotiation.cc:769] Sending connection context
0812 06:56:43.099046 (+ 298us) client_negotiation.cc:240] Negotiation successful
0812 06:56:43.099548 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":383,"threads_started":1}
I20260812 06:56:43.129884 7710 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:40059
I20260812 06:56:43.129913 7935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:40059 every 8 connection(s)
I20260812 06:56:43.130585 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:43.115566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:39825 (local address 127.26.226.62:36507)
0812 06:56:43.115808 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:43.115816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:43.116446 (+ 630us) server_negotiation.cc:407] Connection header received
0812 06:56:43.116725 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:43.116773 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:43.116878 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:43.116951 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:43.118350 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.119372 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.121614 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.122192 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.122320 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:43.124220 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:43.124226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:43.124231 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:43.124269 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:43.127544 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:43.128503 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:43.128509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:43.128515 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:43.128697 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:43.129126 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:43.129129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:43.129131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:43.129388 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:43.129545 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:43.129999 (+ 454us) server_negotiation.cc:299] Negotiation successful
0812 06:56:43.130224 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:56:43.130679 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:43.115587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36507 (local address 127.26.226.1:39825)
0812 06:56:43.116337 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:43.116353 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:43.116521 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:43.117134 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:43.117137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:43.117825 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:43.118202 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:43.118214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.119545 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.119549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:43.121440 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:43.121461 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.122401 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.122408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:43.122588 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:43.123286 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:43.123303 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:43.123997 (+ 694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:43.127730 (+ 3733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:43.127738 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:43.127746 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:43.128328 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:43.128841 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:43.128846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:43.128850 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:43.128995 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:43.129512 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:43.129517 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:43.129729 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 06:56:43.129876 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 06:56:43.130224 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":365,"threads_started":1}
I20260812 06:56:43.131417 7710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:43.133817 7936 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36507
I20260812 06:56:43.134065 7936 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:43.134728 7936 heartbeater.cc:511] Master 127.26.226.62:36507 requested a full tablet report, sending...
I20260812 06:56:43.135671 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 7710
I20260812 06:56:43.135867 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:43.136559 27528 external_mini_cluster.cc:1428] Setting key 0bd66742d39a60339e0dd17399f5bcc0
I20260812 06:56:43.136854 7581 ts_manager.cc:194] Registered new tserver with Master: 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059)
I20260812 06:56:43.138526 7581 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:45869
WARNING: no policy specified for oryx/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab.
Entry for principal oryx/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab.
I20260812 06:56:43.196154 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36507
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab
--principal=oryx/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:43.545920 7943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:43.546269 7943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:43.546447 7943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:43.553880 7943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:43.554046 7943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:43.554198 7943 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:56:43.554342 7943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:43.563796 7943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab
--principal=oryx/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.226.62:36507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:43.565073 7943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:43.566411 7943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:43.579195 7953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:43.579227 7955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:43.579360 7957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:43.580441 7943 server_base.cc:1034] running on GCE node
I20260812 06:56:43.581053 7943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:43.581977 7943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:43.583210 7943 hybrid_clock.cc:648] HybridClock initialized: now 1786517803583144 us; error 32 us; skew 500 ppm
I20260812 06:56:43.583554 7943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:43 dist-test-slave-t3q3 krb5kdc[7515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517803, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:43.586879 7943 init.cc:377] Logged in from keytab as oryx/127.26.226.2@KRBTEST.COM (short username oryx)
I20260812 06:56:43.588341 7943 webserver.cc:460] Webserver started at http://127.26.226.2:36515/ using document root <none> and password file <none>
I20260812 06:56:43.589013 7943 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:43.589102 7943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:43.589375 7943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:43.592332 7943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "933197931f264cf1b228fdcdb0229242"
format_stamp: "Formatted at 2026-08-12 06:56:43 on dist-test-slave-t3q3"
server_key: "3686600f67536a755e37a7c35b055517"
server_key_iv: "78db37c06eae55622ef7dadefcc37850"
server_key_version: "encryptionkey@0"
I20260812 06:56:43.593118 7943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "933197931f264cf1b228fdcdb0229242"
format_stamp: "Formatted at 2026-08-12 06:56:43 on dist-test-slave-t3q3"
server_key: "3686600f67536a755e37a7c35b055517"
server_key_iv: "78db37c06eae55622ef7dadefcc37850"
server_key_version: "encryptionkey@0"
I20260812 06:56:43.598486 7943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:56:43.602192 7971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:43.603590 7943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:56:43.603744 7943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "933197931f264cf1b228fdcdb0229242"
format_stamp: "Formatted at 2026-08-12 06:56:43 on dist-test-slave-t3q3"
server_key: "3686600f67536a755e37a7c35b055517"
server_key_iv: "78db37c06eae55622ef7dadefcc37850"
server_key_version: "encryptionkey@0"
I20260812 06:56:43.604009 7943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:43.622642 7943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:43.627014 7943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:43.627254 7943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:43.630211 7943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:43.630297 7943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:43.630498 7943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:43.630563 7943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:56:43 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517803, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:43.656404 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:43.630747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:49011 (local address 127.26.226.62:36507)
0812 06:56:43.631059 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:43.631069 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:43.632598 (+ 1529us) server_negotiation.cc:407] Connection header received
0812 06:56:43.635040 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:43.635049 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:43.635166 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:43.635309 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:43.637891 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.639013 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.641655 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.642209 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.642366 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:43.647902 (+ 5536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:43.647907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:43.647911 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:43.647971 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:43.652263 (+ 4292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:43.653648 (+ 1385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:43.653652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:43.653654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:43.653795 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:43.654406 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:43.654413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:43.654419 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:43.654805 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:43.654988 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:43.655770 (+ 782us) server_negotiation.cc:299] Negotiation successful
0812 06:56:43.656012 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260812 06:56:43.657971 7988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:43.631461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36507 (local address 127.26.226.2:49011)
0812 06:56:43.632391 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:43.632449 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:43.634637 (+ 2188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:43.635848 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:43.635856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:43.636722 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:43.637682 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:43.637718 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.639219 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.639227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:43.641459 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:43.641492 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.642497 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.642507 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:43.642994 (+ 487us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:43.643870 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:43.643912 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:43.647662 (+ 3750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:43.652454 (+ 4792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:43.652473 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:43.652497 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:43.653437 (+ 940us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:43.653975 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:43.653980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:43.653985 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:43.654204 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:43.654956 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:43.654965 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:43.655351 (+ 386us) client_negotiation.cc:769] Sending connection context
0812 06:56:43.655644 (+ 293us) client_negotiation.cc:240] Negotiation successful
0812 06:56:43.656249 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":414,"threads_started":1}
I20260812 06:56:43.682240 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:43.665255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:45885 (local address 127.26.226.62:36507)
0812 06:56:43.665533 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:43.665541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:43.665693 (+ 152us) server_negotiation.cc:407] Connection header received
0812 06:56:43.665869 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:43.665878 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:43.665986 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:43.666120 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:43.667101 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.668554 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.671239 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.671902 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.672063 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:43.674413 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:43.674422 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:43.674431 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:43.674510 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:43.678077 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:43.679227 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:43.679234 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:43.679239 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:43.679428 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:43.679945 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:43.679954 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:43.679960 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:43.680559 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:43.680753 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:43.681514 (+ 761us) server_negotiation.cc:299] Negotiation successful
0812 06:56:43.681810 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 06:56:43.682647 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:43.664880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36507 (local address 127.26.226.2:45885)
0812 06:56:43.665572 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:43.665593 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:43.665730 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:43.666155 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:43.666158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:43.666641 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:43.666938 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:43.666947 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.668744 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.668752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:43.670954 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:43.670979 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:43.672158 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:43.672166 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:43.672370 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:43.673286 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:43.673317 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:43.674197 (+ 880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:43.678231 (+ 4034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:43.678241 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:43.678245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:43.678997 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:43.679568 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:43.679573 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:43.679577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:43.679792 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:43.680828 (+ 1036us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:43.680839 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:43.681147 (+ 308us) client_negotiation.cc:769] Sending connection context
0812 06:56:43.681349 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 06:56:43.681860 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":456,"threads_started":1}
I20260812 06:56:43.688179 7943 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:34997
I20260812 06:56:43.688259 8175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:34997 every 8 connection(s)
I20260812 06:56:43.689412 7943 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:43.691533 8178 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36507
I20260812 06:56:43.691787 8178 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:43.692539 8178 heartbeater.cc:511] Master 127.26.226.62:36507 requested a full tablet report, sending...
I20260812 06:56:43.693951 7581 ts_manager.cc:194] Registered new tserver with Master: 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997)
I20260812 06:56:43.694814 7581 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:49011
I20260812 06:56:43.695617 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 7943
I20260812 06:56:43.695827 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:43.696362 27528 external_mini_cluster.cc:1428] Setting key 1cac4a254d79405f741d8de9712f7f3d
WARNING: no policy specified for oryx/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab.
Entry for principal oryx/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab.
I20260812 06:56:43.758724 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36507
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab
--principal=oryx/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:44.114112 8185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:44.114426 8185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:44.114621 8185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:44.121927 8185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:44.122063 8185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:44.122213 8185 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:56:44.122366 8185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:44.134331 8185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab
--principal=oryx/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.226.62:36507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:44.135774 8185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:44.137094 8185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:56:44.141270 7936 heartbeater.cc:503] Master 127.26.226.62:36507 was elected leader, sending a full tablet report...
W20260812 06:56:44.150111 8200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:44.150177 8194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:44.150261 8195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:44.151345 8185 server_base.cc:1034] running on GCE node
I20260812 06:56:44.151953 8185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:44.152953 8185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:44.154224 8185 hybrid_clock.cc:648] HybridClock initialized: now 1786517804154153 us; error 33 us; skew 500 ppm
I20260812 06:56:44.154567 8185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:44 dist-test-slave-t3q3 krb5kdc[7515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517804, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:44.157979 8185 init.cc:377] Logged in from keytab as oryx/127.26.226.3@KRBTEST.COM (short username oryx)
I20260812 06:56:44.159469 8185 webserver.cc:460] Webserver started at http://127.26.226.3:34983/ using document root <none> and password file <none>
I20260812 06:56:44.160208 8185 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:44.160308 8185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:44.160656 8185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:44.163657 8185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "58767d36b6704b18bac8a98076f623cb"
format_stamp: "Formatted at 2026-08-12 06:56:44 on dist-test-slave-t3q3"
server_key: "33a779a479d63f32c1aa4868d54ee29a"
server_key_iv: "a20098db04f6021408cf1543038a3eb9"
server_key_version: "encryptionkey@0"
I20260812 06:56:44.164489 8185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "58767d36b6704b18bac8a98076f623cb"
format_stamp: "Formatted at 2026-08-12 06:56:44 on dist-test-slave-t3q3"
server_key: "33a779a479d63f32c1aa4868d54ee29a"
server_key_iv: "a20098db04f6021408cf1543038a3eb9"
server_key_version: "encryptionkey@0"
I20260812 06:56:44.169472 8185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 06:56:44.173020 8216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:44.174324 8185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:44.174494 8185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "58767d36b6704b18bac8a98076f623cb"
format_stamp: "Formatted at 2026-08-12 06:56:44 on dist-test-slave-t3q3"
server_key: "33a779a479d63f32c1aa4868d54ee29a"
server_key_iv: "a20098db04f6021408cf1543038a3eb9"
server_key_version: "encryptionkey@0"
I20260812 06:56:44.174727 8185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:44.195968 8185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:44.201041 8185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:44.201354 8185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:44.204473 8185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:44.204663 8185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:44.204835 8185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:44.204926 8185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:56:44 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517804, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:44.232858 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:44.204757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:50863 (local address 127.26.226.62:36507)
0812 06:56:44.205317 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:44.205324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:44.206624 (+ 1300us) server_negotiation.cc:407] Connection header received
0812 06:56:44.210094 (+ 3470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:44.210101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:44.210228 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:44.210370 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:44.213411 (+ 3041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.214358 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.217882 (+ 3524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.218335 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.218496 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:44.224850 (+ 6354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:44.224855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:44.224859 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:44.224900 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:44.228649 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:44.229921 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:44.229925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:44.229928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:44.230088 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:44.230733 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:44.230737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:44.230739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:44.231068 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:44.231189 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:44.232405 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0812 06:56:44.232573 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":270,"threads_started":1}
I20260812 06:56:44.235251 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:44.205439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36507 (local address 127.26.226.3:50863)
0812 06:56:44.206421 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:44.206486 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:44.209727 (+ 3241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:44.210708 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:44.210723 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:44.211814 (+ 1091us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:44.213073 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:44.213106 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.214597 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.214607 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:44.217697 (+ 3090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:44.217721 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.218677 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.218689 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:44.219125 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:44.220191 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:44.220240 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:44.224504 (+ 4264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:44.228822 (+ 4318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:44.228838 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:44.228864 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:44.229736 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:44.230222 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:44.230230 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:44.230237 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:44.230596 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:44.231208 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:44.231221 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:44.231852 (+ 631us) client_negotiation.cc:769] Sending connection context
0812 06:56:44.232305 (+ 453us) client_negotiation.cc:240] Negotiation successful
0812 06:56:44.233109 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":415,"threads_started":1}
I20260812 06:56:44.259593 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:44.241843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:34871 (local address 127.26.226.62:36507)
0812 06:56:44.242078 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:44.242087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:44.242855 (+ 768us) server_negotiation.cc:407] Connection header received
0812 06:56:44.242967 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:44.242975 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:44.243138 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:44.243334 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:44.245224 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.246239 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.248415 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.249390 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.249566 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:44.251517 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:44.251528 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:44.251537 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:44.251594 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:44.255887 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:44.256877 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:44.256882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:44.256886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:44.257023 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:44.257662 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:44.257672 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:44.257679 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:44.258201 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:44.258426 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:44.259158 (+ 732us) server_negotiation.cc:299] Negotiation successful
0812 06:56:44.259356 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 06:56:44.260437 8340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:44.241796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36507 (local address 127.26.226.3:34871)
0812 06:56:44.242615 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:44.242643 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:44.242814 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:44.243358 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:44.243366 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:44.244359 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:44.245024 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:44.245043 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.246403 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.246408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:44.248192 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:44.248210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.249642 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.249655 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:44.249831 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:44.250500 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:44.250518 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:44.251279 (+ 761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:44.256052 (+ 4773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:44.256057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:44.256093 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:44.256725 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:44.257150 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:44.257165 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:44.257168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:44.257382 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:44.258512 (+ 1130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:44.258520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:44.258815 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 06:56:44.259050 (+ 235us) client_negotiation.cc:240] Negotiation successful
0812 06:56:44.259557 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":399,"threads_started":1}
I20260812 06:56:44.271842 8185 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:41031
I20260812 06:56:44.271885 8416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:41031 every 8 connection(s)
I20260812 06:56:44.273272 8185 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:44.275472 8417 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36507
I20260812 06:56:44.275744 8417 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:44.276526 8417 heartbeater.cc:511] Master 127.26.226.62:36507 requested a full tablet report, sending...
I20260812 06:56:44.277582 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 8185
I20260812 06:56:44.277778 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:44.278041 7581 ts_manager.cc:194] Registered new tserver with Master: 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031)
I20260812 06:56:44.278359 27528 external_mini_cluster.cc:1428] Setting key 198d538e53fc1518eb806242ff64c8b0
I20260812 06:56:44.279026 7581 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:50863
I20260812 06:56:44.281657 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:56:44 dist-test-slave-t3q3 krb5kdc[7515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517804, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:56:44 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:44 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:44.697896 8178 heartbeater.cc:503] Master 127.26.226.62:36507 was elected leader, sending a full tablet report...
I20260812 06:56:45.282128 8417 heartbeater.cc:503] Master 127.26.226.62:36507 was elected leader, sending a full tablet report...
W20260812 06:56:45.458056 8434 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.117s user 0.006s sys 0.014s
W20260812 06:56:45.459542 8227 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:44.334780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34862 (local address 127.26.226.62:36507)
0812 06:56:44.335023 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:44.335030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:44.335349 (+ 319us) server_negotiation.cc:407] Connection header received
0812 06:56:44.335603 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:44.335607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:44.335673 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:44.335788 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:44.337331 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.338083 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.339364 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.339773 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.339919 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.459170 (+1119251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34862: BlockingRecv error: recv got EOF from 127.0.0.1:34862 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 12 06:56:45 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517804, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:45.486949 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.466016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34868 (local address 127.26.226.62:36507)
0812 06:56:45.466287 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:45.466294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:45.466654 (+ 360us) server_negotiation.cc:407] Connection header received
0812 06:56:45.466935 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:45.466946 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:45.467047 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:45.467206 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:45.469214 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.470617 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.472930 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.473709 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.473887 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.479754 (+ 5867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:45.479766 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:45.479778 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:45.479831 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:45.484096 (+ 4265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:45.485102 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:45.485111 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:45.485118 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:45.485310 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:45.485758 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:45.485762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:45.485764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:45.486045 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:45.486298 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:45.486547 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 06:56:45.486734 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 06:56:45.491371 7578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:45.498422 7578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:45.530404 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.511202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.26.226.2:34997)
0812 06:56:45.521063 (+ 9861us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:45.521075 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:45.521111 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:56:45.521248 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:45.521257 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:45.521545 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:45.521847 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:45.522830 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.524616 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.527559 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.529170 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.529340 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.529594 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:45.529898 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 06:56:45.530068 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9695,"thread_start_us":9561,"threads_started":1}
I20260812 06:56:45.530501 8453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.510982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:34997 (local address 127.0.0.1:56736)
0812 06:56:45.512559 (+ 1577us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:45.512607 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:45.512810 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:45.521845 (+ 9035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:45.521857 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:45.521905 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:45.522530 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:45.522548 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.524757 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.524762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:45.527408 (+ 2646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:45.527424 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.529411 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.529419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:45.529629 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.529684 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 06:56:45.529853 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 06:56:45.530066 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1238,"mutex_wait_us":61,"thread_start_us":10174,"threads_started":1}
I20260812 06:56:45.530927 8451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.511685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:40059 (local address 127.0.0.1:35658)
0812 06:56:45.521473 (+ 9788us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:45.521494 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:45.522522 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:45.523416 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:45.523420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:45.523446 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:45.523948 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:45.523968 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.525947 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.525954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:45.527818 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:45.527834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.530035 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.530041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:45.530220 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.530257 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 06:56:45.530346 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 06:56:45.530492 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9626,"mutex_wait_us":2,"thread_start_us":585,"threads_started":1}
I20260812 06:56:45.531512 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.511997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.26.226.1:40059)
0812 06:56:45.520712 (+ 8715us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:45.520726 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:45.522126 (+ 1400us) server_negotiation.cc:407] Connection header received
0812 06:56:45.522720 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:45.522727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:45.522994 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:45.523192 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:45.524286 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.525815 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.528015 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.529489 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.529658 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.529881 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:45.531045 (+ 1164us) server_negotiation.cc:299] Negotiation successful
0812 06:56:45.531224 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8478,"thread_start_us":8315,"threads_started":1}
I20260812 06:56:45.533123 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.521577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:41031 (local address 127.0.0.1:56994)
0812 06:56:45.524326 (+ 2749us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:45.524340 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:45.524480 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:45.525098 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:45.525104 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:45.525137 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:45.525575 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:45.525585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.527449 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.527454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:45.529825 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:45.529841 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.532304 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.532310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:45.532497 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.532541 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 06:56:45.532649 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 06:56:45.532791 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2635,"thread_start_us":2410,"threads_started":1}
I20260812 06:56:45.533197 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.521828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56994 (local address 127.26.226.3:41031)
0812 06:56:45.522558 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:45.522568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:45.524440 (+ 1872us) server_negotiation.cc:407] Connection header received
0812 06:56:45.524646 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:45.524651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:45.524885 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:45.525086 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:45.525768 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.527302 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.530048 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.532034 (+ 1986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.532254 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.532455 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:45.532721 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 06:56:45.532905 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":454,"threads_started":1}
I20260812 06:56:45.534981 8058 tablet_service.cc:1467] Processing CreateTablet for tablet c4a1e5fd38cc4a42b98ef17862277c7e (DEFAULT_TABLE table=test-table [id=90f5a385b1774767b819960d4fa71f60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:45.535128 7830 tablet_service.cc:1467] Processing CreateTablet for tablet c4a1e5fd38cc4a42b98ef17862277c7e (DEFAULT_TABLE table=test-table [id=90f5a385b1774767b819960d4fa71f60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:45.536334 7830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a1e5fd38cc4a42b98ef17862277c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:45.536381 8058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a1e5fd38cc4a42b98ef17862277c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:45.538738 8302 tablet_service.cc:1467] Processing CreateTablet for tablet c4a1e5fd38cc4a42b98ef17862277c7e (DEFAULT_TABLE table=test-table [id=90f5a385b1774767b819960d4fa71f60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:45.540130 8302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a1e5fd38cc4a42b98ef17862277c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:45.547761 8469 tablet_bootstrap.cc:492] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: Bootstrap starting.
I20260812 06:56:45.551672 8469 tablet_bootstrap.cc:654] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:45.551788 8471 tablet_bootstrap.cc:492] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: Bootstrap starting.
I20260812 06:56:45.553227 8469 log.cc:824] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:45.555488 8471 tablet_bootstrap.cc:654] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:45.556262 8474 tablet_bootstrap.cc:492] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: Bootstrap starting.
I20260812 06:56:45.556615 8469 tablet_bootstrap.cc:492] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: No bootstrap required, opened a new log
I20260812 06:56:45.556821 8471 log.cc:824] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:45.556900 8469 ts_tablet_manager.cc:1397] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 06:56:45.559173 8474 tablet_bootstrap.cc:654] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:45.559953 8471 tablet_bootstrap.cc:492] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: No bootstrap required, opened a new log
I20260812 06:56:45.560287 8471 ts_tablet_manager.cc:1397] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260812 06:56:45.560474 8474 log.cc:824] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:45.567462 8469 raft_consensus.cc:348] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } }
I20260812 06:56:45.567942 8469 raft_consensus.cc:374] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:45.568054 8469 raft_consensus.cc:729] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7436a98362ff4489a0255322b0a0753e, State: Initialized, Role: FOLLOWER
I20260812 06:56:45.568835 8469 consensus_queue.cc:260] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } }
I20260812 06:56:45.570600 8469 ts_tablet_manager.cc:1428] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260812 06:56:45.571080 8474 tablet_bootstrap.cc:492] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: No bootstrap required, opened a new log
I20260812 06:56:45.571393 8474 ts_tablet_manager.cc:1397] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260812 06:56:45.571128 8471 raft_consensus.cc:348] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } }
I20260812 06:56:45.571584 8471 raft_consensus.cc:374] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:45.571696 8471 raft_consensus.cc:729] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933197931f264cf1b228fdcdb0229242, State: Initialized, Role: FOLLOWER
I20260812 06:56:45.572324 8471 consensus_queue.cc:260] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } }
I20260812 06:56:45.574092 8471 ts_tablet_manager.cc:1428] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260812 06:56:45.580674 8474 raft_consensus.cc:348] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } }
I20260812 06:56:45.581043 8474 raft_consensus.cc:374] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:45.581151 8474 raft_consensus.cc:729] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58767d36b6704b18bac8a98076f623cb, State: Initialized, Role: FOLLOWER
I20260812 06:56:45.581761 8474 consensus_queue.cc:260] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } }
I20260812 06:56:45.583302 8474 ts_tablet_manager.cc:1428] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
W20260812 06:56:45.635051 7939 tablet.cc:2369] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:56:45.692575 8179 tablet.cc:2369] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:56:45.776257 8418 tablet.cc:2369] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:45.824283 8480 raft_consensus.cc:482] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:45.824570 8480 raft_consensus.cc:504] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } }
I20260812 06:56:45.826339 8480 leader_election.cc:288] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031), 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059)
I20260812 06:56:45.833793 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.826693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:46277 (local address 127.26.226.3:41031)
0812 06:56:45.827097 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:45.827107 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:45.828304 (+ 1197us) server_negotiation.cc:407] Connection header received
0812 06:56:45.828539 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:45.828546 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:45.828630 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:45.828727 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:45.829330 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.830076 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.831924 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.832827 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.833022 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.833124 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:45.833407 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 06:56:45.833533 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208}
I20260812 06:56:45.834090 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.826995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:48921 (local address 127.26.226.1:40059)
0812 06:56:45.827224 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:45.827232 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:45.828437 (+ 1205us) server_negotiation.cc:407] Connection header received
0812 06:56:45.828651 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:45.828655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:45.828734 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:45.828833 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:45.829522 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.830230 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.832007 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.832854 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.833065 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.833154 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:45.833457 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 06:56:45.833600 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 06:56:45.834082 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.826887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:40059 (local address 127.26.226.2:48921)
0812 06:56:45.828315 (+ 1428us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:45.828333 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:45.828493 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:45.828854 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:45.828859 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:45.828899 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:45.829347 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:45.829356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.830373 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.830380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:45.831843 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:45.831856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.833120 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.833127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:45.833296 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.833319 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:56:45.833428 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 06:56:45.833561 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1279,"mutex_wait_us":23,"thread_start_us":902,"threads_started":1}
I20260812 06:56:45.834053 8482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.826765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:41031 (local address 127.26.226.2:46277)
0812 06:56:45.828149 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:45.828181 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:45.828395 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:45.828756 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:45.828761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:45.828792 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:45.829177 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:45.829188 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.830226 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.830232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:45.831764 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:45.831780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.833067 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.833074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:45.833243 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.833267 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:56:45.833378 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 06:56:45.833511 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"mutex_wait_us":1,"thread_start_us":855,"threads_started":1}
I20260812 06:56:45.835366 8346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a1e5fd38cc4a42b98ef17862277c7e" candidate_uuid: "933197931f264cf1b228fdcdb0229242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58767d36b6704b18bac8a98076f623cb" is_pre_election: true
I20260812 06:56:45.835414 7859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a1e5fd38cc4a42b98ef17862277c7e" candidate_uuid: "933197931f264cf1b228fdcdb0229242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7436a98362ff4489a0255322b0a0753e" is_pre_election: true
I20260812 06:56:45.835901 7859 raft_consensus.cc:2454] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 933197931f264cf1b228fdcdb0229242 in term 0.
I20260812 06:56:45.835899 8346 raft_consensus.cc:2454] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 933197931f264cf1b228fdcdb0229242 in term 0.
I20260812 06:56:45.836611 7981 leader_election.cc:302] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7436a98362ff4489a0255322b0a0753e, 933197931f264cf1b228fdcdb0229242; no voters:
I20260812 06:56:45.837102 8480 raft_consensus.cc:2781] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:45.837260 8480 raft_consensus.cc:482] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:45.837383 8480 raft_consensus.cc:3037] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:45.839586 8480 raft_consensus.cc:504] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } }
I20260812 06:56:45.840273 8480 leader_election.cc:288] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [CANDIDATE]: Term 1 election: Requested vote from peers 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031), 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059)
I20260812 06:56:45.840695 8346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a1e5fd38cc4a42b98ef17862277c7e" candidate_uuid: "933197931f264cf1b228fdcdb0229242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58767d36b6704b18bac8a98076f623cb"
I20260812 06:56:45.840745 7859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a1e5fd38cc4a42b98ef17862277c7e" candidate_uuid: "933197931f264cf1b228fdcdb0229242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7436a98362ff4489a0255322b0a0753e"
I20260812 06:56:45.840924 8346 raft_consensus.cc:3037] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:45.840934 7859 raft_consensus.cc:3037] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:45.844209 7859 raft_consensus.cc:2454] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 933197931f264cf1b228fdcdb0229242 in term 1.
I20260812 06:56:45.844209 8346 raft_consensus.cc:2454] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 933197931f264cf1b228fdcdb0229242 in term 1.
I20260812 06:56:45.844704 7980 leader_election.cc:302] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 58767d36b6704b18bac8a98076f623cb, 7436a98362ff4489a0255322b0a0753e, 933197931f264cf1b228fdcdb0229242; no voters:
I20260812 06:56:45.845073 8480 raft_consensus.cc:2781] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:45.845549 8480 raft_consensus.cc:686] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 1 LEADER]: Becoming Leader. State: Replica: 933197931f264cf1b228fdcdb0229242, State: Running, Role: LEADER
I20260812 06:56:45.846028 8480 consensus_queue.cc:237] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } }
I20260812 06:56:45.850795 7578 catalog_manager.cc:5495] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 reported cstate change: term changed from 0 to 1, leader changed from <none> to 933197931f264cf1b228fdcdb0229242 (127.26.226.2). New cstate: current_term: 1 leader_uuid: "933197931f264cf1b228fdcdb0229242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } health_report { overall_health: UNKNOWN } } }
I20260812 06:56:45.889144 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:45.883971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56742 (local address 127.26.226.2:34997)
0812 06:56:45.884237 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:45.884241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:45.884259 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:56:45.884420 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:45.884424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:45.884514 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:45.884639 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:45.884944 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.885633 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.886934 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:45.887364 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:45.887461 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:45.887866 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:45.888490 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:45.888601 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:45.888749 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 06:56:45.888819 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260812 06:56:45.898203 7859 raft_consensus.cc:1261] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Refusing update from remote peer 933197931f264cf1b228fdcdb0229242: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:45.898203 8346 raft_consensus.cc:1261] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Refusing update from remote peer 933197931f264cf1b228fdcdb0229242: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:45.899534 8489 consensus_queue.cc:1035] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:45.899950 8480 consensus_queue.cc:1035] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:45.915802 8494 mvcc.cc:205] Tried to move back new op lower bound from 7317576932946587648 to 7317576932748951552. Current Snapshot: MvccSnapshot[applied={T|T < 7317576932946587648}]
I20260812 06:56:45.928730 7581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:45.929029 7581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:45.932809 7581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837: Sending DeleteTablet for 3 replicas of tablet c4a1e5fd38cc4a42b98ef17862277c7e
I20260812 06:56:45.934021 8302 tablet_service.cc:1514] Processing DeleteTablet for tablet c4a1e5fd38cc4a42b98ef17862277c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:45 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:56994
I20260812 06:56:45.934021 7830 tablet_service.cc:1514] Processing DeleteTablet for tablet c4a1e5fd38cc4a42b98ef17862277c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:45 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:35658
I20260812 06:56:45.934073 8058 tablet_service.cc:1514] Processing DeleteTablet for tablet c4a1e5fd38cc4a42b98ef17862277c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:45 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:56736
I20260812 06:56:45.936267 7581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:45.937470 7581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:45.945613 8503 tablet_replica.cc:330] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: stopping tablet replica
I20260812 06:56:45.946425 8503 raft_consensus.cc:2229] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:45.947176 8503 raft_consensus.cc:2258] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:45.948081 8299 tablet_service.cc:1467] Processing CreateTablet for tablet d6fa08a3b91045b1b74e706b31a811c4 (DEFAULT_TABLE table=test-table [id=ae094850f2834ec29eebef048e89f968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:45.948096 8054 tablet_service.cc:1467] Processing CreateTablet for tablet d6fa08a3b91045b1b74e706b31a811c4 (DEFAULT_TABLE table=test-table [id=ae094850f2834ec29eebef048e89f968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:45.948156 7830 tablet_service.cc:1467] Processing CreateTablet for tablet d6fa08a3b91045b1b74e706b31a811c4 (DEFAULT_TABLE table=test-table [id=ae094850f2834ec29eebef048e89f968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:45.948657 7830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fa08a3b91045b1b74e706b31a811c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:45.948719 8299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fa08a3b91045b1b74e706b31a811c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:45.948724 8054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fa08a3b91045b1b74e706b31a811c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:45.955516 8474 tablet_bootstrap.cc:492] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb: Bootstrap starting.
I20260812 06:56:45.956269 8471 tablet_bootstrap.cc:492] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242: Bootstrap starting.
I20260812 06:56:45.956926 8469 tablet_bootstrap.cc:492] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e: Bootstrap starting.
I20260812 06:56:45.957353 8474 tablet_bootstrap.cc:654] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:45.958019 8471 tablet_bootstrap.cc:654] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:45.958626 8469 tablet_bootstrap.cc:654] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:45.964978 8503 ts_tablet_manager.cc:1905] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:45.970578 8508 tablet_replica.cc:330] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: stopping tablet replica
I20260812 06:56:45.971141 8508 raft_consensus.cc:2229] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:45.971552 8507 tablet_replica.cc:330] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: stopping tablet replica
I20260812 06:56:45.971876 8508 raft_consensus.cc:2258] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:45.972011 8507 raft_consensus.cc:2229] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:45.972628 8507 raft_consensus.cc:2258] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:45.974910 8469 tablet_bootstrap.cc:492] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e: No bootstrap required, opened a new log
I20260812 06:56:45.975065 8471 tablet_bootstrap.cc:492] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242: No bootstrap required, opened a new log
I20260812 06:56:45.975082 8469 ts_tablet_manager.cc:1397] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.009s
I20260812 06:56:45.975240 8471 ts_tablet_manager.cc:1397] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.007s
I20260812 06:56:45.975952 8469 raft_consensus.cc:348] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:45.976233 8469 raft_consensus.cc:374] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:45.976107 8471 raft_consensus.cc:348] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:45.976325 8469 raft_consensus.cc:729] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7436a98362ff4489a0255322b0a0753e, State: Initialized, Role: FOLLOWER
I20260812 06:56:45.976356 8471 raft_consensus.cc:374] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:45.976465 8471 raft_consensus.cc:729] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933197931f264cf1b228fdcdb0229242, State: Initialized, Role: FOLLOWER
I20260812 06:56:45.976711 8469 consensus_queue.cc:260] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:45.976734 8471 consensus_queue.cc:260] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:45.977396 8469 ts_tablet_manager.cc:1428] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:45.977413 8471 ts_tablet_manager.cc:1428] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 06:56:45.979619 8507 ts_tablet_manager.cc:1905] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:45.982710 8508 ts_tablet_manager.cc:1905] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:45.983016 8474 tablet_bootstrap.cc:492] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb: No bootstrap required, opened a new log
I20260812 06:56:45.983220 8474 ts_tablet_manager.cc:1397] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.004s
I20260812 06:56:45.983927 8474 raft_consensus.cc:348] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:45.984181 8474 raft_consensus.cc:374] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:45.984313 8474 raft_consensus.cc:729] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58767d36b6704b18bac8a98076f623cb, State: Initialized, Role: FOLLOWER
I20260812 06:56:45.984630 8474 consensus_queue.cc:260] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:45.985287 8474 ts_tablet_manager.cc:1428] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:45.987286 8507 ts_tablet_manager.cc:1918] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:45.987449 8507 log.cc:1196] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/c4a1e5fd38cc4a42b98ef17862277c7e
I20260812 06:56:45.987995 8507 ts_tablet_manager.cc:1939] T c4a1e5fd38cc4a42b98ef17862277c7e P 58767d36b6704b18bac8a98076f623cb: Deleting consensus metadata
I20260812 06:56:45.990067 8508 ts_tablet_manager.cc:1918] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:45.990235 8508 log.cc:1196] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/c4a1e5fd38cc4a42b98ef17862277c7e
I20260812 06:56:45.990795 8508 ts_tablet_manager.cc:1939] T c4a1e5fd38cc4a42b98ef17862277c7e P 933197931f264cf1b228fdcdb0229242: Deleting consensus metadata
I20260812 06:56:45.992206 8503 ts_tablet_manager.cc:1918] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:45.992389 8503 log.cc:1196] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/c4a1e5fd38cc4a42b98ef17862277c7e
I20260812 06:56:45.992810 7553 catalog_manager.cc:4841] TS 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997): tablet c4a1e5fd38cc4a42b98ef17862277c7e (table test-table [id=90f5a385b1774767b819960d4fa71f60]) successfully deleted
I20260812 06:56:45.992991 8503 ts_tablet_manager.cc:1939] T c4a1e5fd38cc4a42b98ef17862277c7e P 7436a98362ff4489a0255322b0a0753e: Deleting consensus metadata
I20260812 06:56:45.994123 7553 catalog_manager.cc:4841] TS 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031): tablet c4a1e5fd38cc4a42b98ef17862277c7e (table test-table [id=90f5a385b1774767b819960d4fa71f60]) successfully deleted
I20260812 06:56:45.994459 7555 catalog_manager.cc:4841] TS 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059): tablet c4a1e5fd38cc4a42b98ef17862277c7e (table test-table [id=90f5a385b1774767b819960d4fa71f60]) successfully deleted
I20260812 06:56:46.224748 8481 raft_consensus.cc:482] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:46.225018 8481 raft_consensus.cc:504] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.226609 8481 leader_election.cc:288] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059), 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997)
I20260812 06:56:46.235327 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.227164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:47149 (local address 127.26.226.2:34997)
0812 06:56:46.227408 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.227416 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.228355 (+ 939us) server_negotiation.cc:407] Connection header received
0812 06:56:46.228757 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.228765 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.228857 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.228976 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:46.229859 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.230958 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.232929 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.234158 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.234334 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.234415 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.234851 (+ 436us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.234997 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:56:46.235558 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.227202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:49059 (local address 127.26.226.1:40059)
0812 06:56:46.227409 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.227416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.228576 (+ 1160us) server_negotiation.cc:407] Connection header received
0812 06:56:46.228804 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.228808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.228868 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.228969 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:46.229604 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.230729 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.233382 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.234606 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.234798 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.234870 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.235153 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.235287 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 06:56:46.235671 8519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.227781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:40059 (local address 127.26.226.3:49059)
0812 06:56:46.228483 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.228500 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.228673 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.229000 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.229004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.229032 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:46.229412 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.229423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.230902 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.230909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.233214 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.233230 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.234828 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.234834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.234987 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.235009 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.235126 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.235256 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":6,"thread_start_us":462,"threads_started":1}
I20260812 06:56:46.236336 8517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.227425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:34997 (local address 127.26.226.3:47149)
0812 06:56:46.228207 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.228243 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.228482 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.229165 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.229170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.229204 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:46.229671 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.229688 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.231096 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.231102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.232753 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.232767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.234407 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.234415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.234627 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.234663 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.235621 (+ 958us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.235926 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":26,"thread_start_us":517,"threads_started":1}
I20260812 06:56:46.236606 7859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fa08a3b91045b1b74e706b31a811c4" candidate_uuid: "58767d36b6704b18bac8a98076f623cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7436a98362ff4489a0255322b0a0753e" is_pre_election: true
I20260812 06:56:46.236891 7859 raft_consensus.cc:2454] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58767d36b6704b18bac8a98076f623cb in term 0.
I20260812 06:56:46.237388 8221 leader_election.cc:302] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58767d36b6704b18bac8a98076f623cb, 7436a98362ff4489a0255322b0a0753e; no voters:
I20260812 06:56:46.237237 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fa08a3b91045b1b74e706b31a811c4" candidate_uuid: "58767d36b6704b18bac8a98076f623cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933197931f264cf1b228fdcdb0229242" is_pre_election: true
I20260812 06:56:46.237800 8481 raft_consensus.cc:2781] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:46.237742 8093 raft_consensus.cc:2454] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58767d36b6704b18bac8a98076f623cb in term 0.
I20260812 06:56:46.237933 8481 raft_consensus.cc:482] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:46.238051 8481 raft_consensus.cc:3037] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:46.240365 8481 raft_consensus.cc:504] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.241066 8481 leader_election.cc:288] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [CANDIDATE]: Term 1 election: Requested vote from peers 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059), 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997)
I20260812 06:56:46.241439 7859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fa08a3b91045b1b74e706b31a811c4" candidate_uuid: "58767d36b6704b18bac8a98076f623cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7436a98362ff4489a0255322b0a0753e"
I20260812 06:56:46.241643 7859 raft_consensus.cc:3037] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:46.241786 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fa08a3b91045b1b74e706b31a811c4" candidate_uuid: "58767d36b6704b18bac8a98076f623cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933197931f264cf1b228fdcdb0229242"
I20260812 06:56:46.242002 8093 raft_consensus.cc:3037] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:46.244689 7859 raft_consensus.cc:2454] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58767d36b6704b18bac8a98076f623cb in term 1.
I20260812 06:56:46.245189 8221 leader_election.cc:302] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58767d36b6704b18bac8a98076f623cb, 7436a98362ff4489a0255322b0a0753e; no voters:
I20260812 06:56:46.245388 8093 raft_consensus.cc:2454] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58767d36b6704b18bac8a98076f623cb in term 1.
I20260812 06:56:46.245760 8481 raft_consensus.cc:2781] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:46.246310 8481 raft_consensus.cc:686] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 1 LEADER]: Becoming Leader. State: Replica: 58767d36b6704b18bac8a98076f623cb, State: Running, Role: LEADER
I20260812 06:56:46.246713 8481 consensus_queue.cc:237] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.250468 7581 catalog_manager.cc:5495] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 58767d36b6704b18bac8a98076f623cb (127.26.226.3). New cstate: current_term: 1 leader_uuid: "58767d36b6704b18bac8a98076f623cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } health_report { overall_health: HEALTHY } } }
Aug 12 06:56:46 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517802, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.62@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:46.307976 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.282813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36507 (local address 127.0.0.1:34876)
0812 06:56:46.292913 (+ 10100us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.292934 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.293194 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.293609 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.293614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.294464 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:46.294710 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.294720 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.295900 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.295907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.297057 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.297071 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.297901 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.297908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.298165 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.298937 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:46.298962 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:46.301448 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:46.305045 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:46.305063 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:46.305072 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:46.305547 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:46.305947 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:46.305954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:46.305958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:46.306126 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:46.306755 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:46.306765 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:46.306958 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.307138 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.307461 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9915,"thread_start_us":9752,"threads_started":1}
I20260812 06:56:46.307991 8539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.282843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.26.226.62:36507)
0812 06:56:46.292616 (+ 9773us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.292625 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.293073 (+ 448us) server_negotiation.cc:407] Connection header received
0812 06:56:46.293357 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.293368 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.293442 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.293580 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:46.294882 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.295735 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.297217 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.297710 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.297855 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.301677 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:46.301683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:46.301687 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:46.301719 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:46.304892 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.305689 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.305694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.305706 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.305825 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.306254 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.306259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.306263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.306601 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:46.306792 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.307240 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.307617 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":9586,"thread_start_us":9343,"threads_started":1}
I20260812 06:56:46.313818 7581 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:34876:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 06:56:46.314957 7581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:46.322749 7830 tablet_service.cc:1467] Processing CreateTablet for tablet d23b875c55f64bb7a4f4306da7310c82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b192f3910ab649aea2696580f8bca48a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:56:46.322991 8058 tablet_service.cc:1467] Processing CreateTablet for tablet d23b875c55f64bb7a4f4306da7310c82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b192f3910ab649aea2696580f8bca48a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:56:46.323514 7830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d23b875c55f64bb7a4f4306da7310c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:46.323339 8302 tablet_service.cc:1467] Processing CreateTablet for tablet d23b875c55f64bb7a4f4306da7310c82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b192f3910ab649aea2696580f8bca48a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:56:46.323622 8058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d23b875c55f64bb7a4f4306da7310c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:46.323971 8302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d23b875c55f64bb7a4f4306da7310c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:46.332098 8474 tablet_bootstrap.cc:492] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: Bootstrap starting.
I20260812 06:56:46.332121 8471 tablet_bootstrap.cc:492] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: Bootstrap starting.
I20260812 06:56:46.333850 8474 tablet_bootstrap.cc:654] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:46.333923 8471 tablet_bootstrap.cc:654] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:46.335350 8469 tablet_bootstrap.cc:492] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: Bootstrap starting.
I20260812 06:56:46.336700 8474 tablet_bootstrap.cc:492] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: No bootstrap required, opened a new log
I20260812 06:56:46.336895 8474 ts_tablet_manager.cc:1397] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 06:56:46.337350 8469 tablet_bootstrap.cc:654] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:46.337615 8474 raft_consensus.cc:348] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.337833 8474 raft_consensus.cc:374] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:46.337934 8474 raft_consensus.cc:729] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58767d36b6704b18bac8a98076f623cb, State: Initialized, Role: FOLLOWER
I20260812 06:56:46.338212 8474 consensus_queue.cc:260] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.338791 8474 ts_tablet_manager.cc:1428] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 06:56:46.338670 8481 tablet_replica.cc:439] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } }
I20260812 06:56:46.338963 8481 tablet_replica.cc:442] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:56:46.339075 8471 tablet_bootstrap.cc:492] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: No bootstrap required, opened a new log
I20260812 06:56:46.339206 8471 ts_tablet_manager.cc:1397] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260812 06:56:46.339872 8471 raft_consensus.cc:348] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.340257 8471 raft_consensus.cc:374] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:46.340371 8471 raft_consensus.cc:729] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933197931f264cf1b228fdcdb0229242, State: Initialized, Role: FOLLOWER
I20260812 06:56:46.340673 8471 consensus_queue.cc:260] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.341305 8471 ts_tablet_manager.cc:1428] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:56:46.341116 8489 tablet_replica.cc:439] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } }
I20260812 06:56:46.341486 8489 tablet_replica.cc:442] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:56:46.343128 8469 tablet_bootstrap.cc:492] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: No bootstrap required, opened a new log
I20260812 06:56:46.343321 8469 ts_tablet_manager.cc:1397] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260812 06:56:46.344195 8469 raft_consensus.cc:348] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.344468 8469 raft_consensus.cc:374] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:46.344578 8469 raft_consensus.cc:729] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7436a98362ff4489a0255322b0a0753e, State: Initialized, Role: FOLLOWER
I20260812 06:56:46.344846 8469 consensus_queue.cc:260] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.345572 8469 ts_tablet_manager.cc:1428] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:46.345405 8479 tablet_replica.cc:439] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } }
I20260812 06:56:46.345801 8479 tablet_replica.cc:442] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:56:46.512696 8479 raft_consensus.cc:482] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:46.512928 8479 raft_consensus.cc:504] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.514554 8479 leader_election.cc:288] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997), 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031)
I20260812 06:56:46.522544 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.514967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:42379 (local address 127.26.226.2:34997)
0812 06:56:46.515219 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.515228 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.515573 (+ 345us) server_negotiation.cc:407] Connection header received
0812 06:56:46.515825 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.515833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.515929 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.516031 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:46.516854 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.517859 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.520404 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.521594 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.521761 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.521838 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.522116 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.522245 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:56:46.522775 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.514925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:34997 (local address 127.26.226.1:42379)
0812 06:56:46.515409 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.515432 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.515630 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.516260 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.516265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.516294 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:46.516642 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.516654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.517995 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.518001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.520240 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.520256 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.521774 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.521779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.521905 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.521960 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.522075 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.522223 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":43,"thread_start_us":286,"threads_started":1}
I20260812 06:56:46.523118 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.515576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:41031 (local address 127.26.226.1:54867)
0812 06:56:46.516195 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.516216 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.516331 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.516664 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.516670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.516705 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:46.517044 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.517053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.518343 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.518348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.520461 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.520472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.522099 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.522105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.522311 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.522343 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.522445 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.522559 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":39,"spinlock_wait_cycles":1920,"thread_start_us":282,"threads_started":1}
I20260812 06:56:46.523131 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.515786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:54867 (local address 127.26.226.3:41031)
0812 06:56:46.516397 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.516401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.516416 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:56:46.516475 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.516479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.516534 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.516620 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:46.517204 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.518217 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.520600 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.521847 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.522013 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.522110 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.522484 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.522649 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":409,"threads_started":1}
I20260812 06:56:46.523459 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d23b875c55f64bb7a4f4306da7310c82" candidate_uuid: "7436a98362ff4489a0255322b0a0753e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933197931f264cf1b228fdcdb0229242" is_pre_election: true
I20260812 06:56:46.523736 8093 raft_consensus.cc:2454] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7436a98362ff4489a0255322b0a0753e in term 0.
I20260812 06:56:46.523777 8346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d23b875c55f64bb7a4f4306da7310c82" candidate_uuid: "7436a98362ff4489a0255322b0a0753e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58767d36b6704b18bac8a98076f623cb" is_pre_election: true
I20260812 06:56:46.524112 8346 raft_consensus.cc:2454] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7436a98362ff4489a0255322b0a0753e in term 0.
I20260812 06:56:46.524282 7742 leader_election.cc:302] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7436a98362ff4489a0255322b0a0753e, 933197931f264cf1b228fdcdb0229242; no voters:
I20260812 06:56:46.524703 8479 raft_consensus.cc:2781] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:46.524806 8479 raft_consensus.cc:482] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:46.524955 8479 raft_consensus.cc:3037] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:46.527208 8479 raft_consensus.cc:504] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.527891 8479 leader_election.cc:288] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [CANDIDATE]: Term 1 election: Requested vote from peers 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997), 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031)
I20260812 06:56:46.528438 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d23b875c55f64bb7a4f4306da7310c82" candidate_uuid: "7436a98362ff4489a0255322b0a0753e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933197931f264cf1b228fdcdb0229242"
I20260812 06:56:46.528679 8093 raft_consensus.cc:3037] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:46.528669 8346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d23b875c55f64bb7a4f4306da7310c82" candidate_uuid: "7436a98362ff4489a0255322b0a0753e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58767d36b6704b18bac8a98076f623cb"
I20260812 06:56:46.528908 8346 raft_consensus.cc:3037] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:46.530730 8093 raft_consensus.cc:2454] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7436a98362ff4489a0255322b0a0753e in term 1.
I20260812 06:56:46.530988 8346 raft_consensus.cc:2454] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7436a98362ff4489a0255322b0a0753e in term 1.
I20260812 06:56:46.531131 7742 leader_election.cc:302] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7436a98362ff4489a0255322b0a0753e, 933197931f264cf1b228fdcdb0229242; no voters:
I20260812 06:56:46.531415 8479 raft_consensus.cc:2781] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:46.531831 8479 raft_consensus.cc:686] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [term 1 LEADER]: Becoming Leader. State: Replica: 7436a98362ff4489a0255322b0a0753e, State: Running, Role: LEADER
I20260812 06:56:46.532297 8479 consensus_queue.cc:237] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.534278 8553 tablet_replica.cc:439] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: TxnStatusTablet state changed. Reason: New leader 7436a98362ff4489a0255322b0a0753e. Latest consensus state: current_term: 1 leader_uuid: "7436a98362ff4489a0255322b0a0753e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } }
I20260812 06:56:46.534734 8553 tablet_replica.cc:442] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:56:46.535501 8557 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 06:56:46.535744 8557 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 06:56:46.535967 7578 catalog_manager.cc:5495] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7436a98362ff4489a0255322b0a0753e (127.26.226.1). New cstate: current_term: 1 leader_uuid: "7436a98362ff4489a0255322b0a0753e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } health_report { overall_health: UNKNOWN } } }
I20260812 06:56:46.592384 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.585543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:40059 (local address 127.0.0.1:35672)
0812 06:56:46.585858 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.585874 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.585997 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.586592 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.586596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.586605 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:56:46.586777 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.586784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.588416 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.588420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.589428 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.589440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.590560 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.590564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.590717 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.590774 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:56:46.591888 (+ 1114us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:56:46.591904 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.591958 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.592140 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260812 06:56:46.592468 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.585489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.26.226.1:40059)
0812 06:56:46.585665 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.585669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.585968 (+ 299us) server_negotiation.cc:407] Connection header received
0812 06:56:46.586208 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.586215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.586343 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.586594 (+ 251us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:46.586924 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.588271 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.589602 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.590402 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.590573 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.590979 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:46.591776 (+ 797us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:46.591936 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.591998 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.592121 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:56:46.597429 8346 raft_consensus.cc:1261] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Refusing update from remote peer 7436a98362ff4489a0255322b0a0753e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:46.597532 8093 raft_consensus.cc:1261] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Refusing update from remote peer 7436a98362ff4489a0255322b0a0753e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:46.598156 8553 consensus_queue.cc:1035] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:46.598531 8479 consensus_queue.cc:1035] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e [LEADER]: Connected to new peer: Peer: permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:46.602097 8489 tablet_replica.cc:439] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: TxnStatusTablet state changed. Reason: New leader 7436a98362ff4489a0255322b0a0753e. Latest consensus state: current_term: 1 leader_uuid: "7436a98362ff4489a0255322b0a0753e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } }
I20260812 06:56:46.602412 8489 tablet_replica.cc:442] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:56:46.606005 8489 tablet_replica.cc:439] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7436a98362ff4489a0255322b0a0753e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } }
I20260812 06:56:46.606266 8489 tablet_replica.cc:442] T d23b875c55f64bb7a4f4306da7310c82 P 933197931f264cf1b228fdcdb0229242: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:56:46.610406 8553 tablet_replica.cc:439] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7436a98362ff4489a0255322b0a0753e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } }
I20260812 06:56:46.610772 8553 tablet_replica.cc:442] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:56:46.621089 8481 tablet_replica.cc:439] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: TxnStatusTablet state changed. Reason: New leader 7436a98362ff4489a0255322b0a0753e. Latest consensus state: current_term: 1 leader_uuid: "7436a98362ff4489a0255322b0a0753e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } }
I20260812 06:56:46.621517 8481 tablet_replica.cc:442] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:56:46.626911 8479 tablet_replica.cc:439] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7436a98362ff4489a0255322b0a0753e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } }
I20260812 06:56:46.627424 8479 tablet_replica.cc:442] T d23b875c55f64bb7a4f4306da7310c82 P 7436a98362ff4489a0255322b0a0753e: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:56:46.633219 8481 tablet_replica.cc:439] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7436a98362ff4489a0255322b0a0753e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } } }
I20260812 06:56:46.633601 8481 tablet_replica.cc:442] T d23b875c55f64bb7a4f4306da7310c82 P 58767d36b6704b18bac8a98076f623cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:56:46.642323 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.629625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.26.226.3:41031)
0812 06:56:46.629916 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.629926 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.636877 (+ 6951us) server_negotiation.cc:407] Connection header received
0812 06:56:46.637134 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.637141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.637236 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.637405 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:46.637910 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.638730 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.640145 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.640574 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.640730 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.641180 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:46.641728 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:46.641908 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.641962 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.642046 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 06:56:46.659993 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.653103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:58509 (local address 127.26.226.1:40059)
0812 06:56:46.653339 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.653346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.653888 (+ 542us) server_negotiation.cc:407] Connection header received
0812 06:56:46.654057 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.654064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.654171 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.654283 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:46.654765 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.656030 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.657502 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.658373 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.658517 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.659043 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:46.659238 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:46.659341 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.659478 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.659586 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:56:46.659994 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.653040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:40059 (local address 127.26.226.3:58509)
0812 06:56:46.653779 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.653802 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.653921 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.654311 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.654314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.654324 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:56:46.654605 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.654613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.656214 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.656218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.657343 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.657356 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.658631 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.658636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.658757 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.658821 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:56:46.659354 (+ 533us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:56:46.659363 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.659423 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.659535 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":414,"threads_started":1}
I20260812 06:56:46.670714 8481 consensus_queue.cc:1035] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:46.672272 8093 raft_consensus.cc:1261] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Refusing update from remote peer 58767d36b6704b18bac8a98076f623cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:46.673028 8481 consensus_queue.cc:1035] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:46.683949 8567 mvcc.cc:205] Tried to move back new op lower bound from 7317576936124010496 to 7317576934390755328. Current Snapshot: MvccSnapshot[applied={T|T < 7317576936124010496}]
I20260812 06:56:46.724545 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.717346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:53863 (local address 127.26.226.3:41031)
0812 06:56:46.717645 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.717651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.717669 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:56:46.717747 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.717754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.717869 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.717977 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:46.718680 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.719835 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.722134 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.722871 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.722972 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.723675 (+ 703us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:46.723857 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:46.723956 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.724157 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.724242 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260812 06:56:46.724964 8594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.716526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:41031 (local address 127.26.226.1:53863)
0812 06:56:46.717347 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.717375 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.717518 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.718002 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.718007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.718021 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:56:46.718449 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.718462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.720020 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.720026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.721965 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.721981 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.723205 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.723216 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.723387 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.723463 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:56:46.723975 (+ 512us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:56:46.723988 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.724114 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.724263 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":2764,"threads_started":1}
I20260812 06:56:46.772773 7578 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:46.773083 7578 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:46.776717 7578 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837: Sending DeleteTablet for 3 replicas of tablet d6fa08a3b91045b1b74e706b31a811c4
I20260812 06:56:46.777531 7830 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fa08a3b91045b1b74e706b31a811c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:46 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:35658
I20260812 06:56:46.777984 8058 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fa08a3b91045b1b74e706b31a811c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:46 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:56736
I20260812 06:56:46.778242 8302 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fa08a3b91045b1b74e706b31a811c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:46 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:56994
I20260812 06:56:46.779373 8597 tablet_replica.cc:330] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e: stopping tablet replica
I20260812 06:56:46.779596 8597 raft_consensus.cc:2229] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:46.779898 8600 tablet_replica.cc:330] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242: stopping tablet replica
I20260812 06:56:46.779898 8597 raft_consensus.cc:2258] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:46.780275 8600 raft_consensus.cc:2229] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:46.780913 8600 raft_consensus.cc:2258] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:46.781169 8597 ts_tablet_manager.cc:1905] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:46.782502 8600 ts_tablet_manager.cc:1905] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:46.789305 8597 ts_tablet_manager.cc:1918] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:56:46.789463 8597 log.cc:1196] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/d6fa08a3b91045b1b74e706b31a811c4
I20260812 06:56:46.789966 8597 ts_tablet_manager.cc:1939] T d6fa08a3b91045b1b74e706b31a811c4 P 7436a98362ff4489a0255322b0a0753e: Deleting consensus metadata
I20260812 06:56:46.791221 7555 catalog_manager.cc:4841] TS 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059): tablet d6fa08a3b91045b1b74e706b31a811c4 (table test-table [id=ae094850f2834ec29eebef048e89f968]) successfully deleted
I20260812 06:56:46.798630 8602 tablet_replica.cc:330] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb: stopping tablet replica
I20260812 06:56:46.799009 8602 raft_consensus.cc:2229] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:46.799731 8602 raft_consensus.cc:2258] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:46.800948 8602 ts_tablet_manager.cc:1905] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:46.807138 8600 ts_tablet_manager.cc:1918] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:56:46.812191 8600 log.cc:1196] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/d6fa08a3b91045b1b74e706b31a811c4
I20260812 06:56:46.812803 8600 ts_tablet_manager.cc:1939] T d6fa08a3b91045b1b74e706b31a811c4 P 933197931f264cf1b228fdcdb0229242: Deleting consensus metadata
I20260812 06:56:46.814244 7553 catalog_manager.cc:4841] TS 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997): tablet d6fa08a3b91045b1b74e706b31a811c4 (table test-table [id=ae094850f2834ec29eebef048e89f968]) successfully deleted
I20260812 06:56:46.815320 8602 ts_tablet_manager.cc:1918] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:56:46.815538 8602 log.cc:1196] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/d6fa08a3b91045b1b74e706b31a811c4
I20260812 06:56:46.816164 8602 ts_tablet_manager.cc:1939] T d6fa08a3b91045b1b74e706b31a811c4 P 58767d36b6704b18bac8a98076f623cb: Deleting consensus metadata
I20260812 06:56:46.817459 7553 catalog_manager.cc:4841] TS 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031): tablet d6fa08a3b91045b1b74e706b31a811c4 (table test-table [id=ae094850f2834ec29eebef048e89f968]) successfully deleted
Aug 12 06:56:46 dist-test-slave-t3q3 krb5kdc[7515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:56:46 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:46 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 06:56:46.884585 8623 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:46.854406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.26.226.62:36507)
0812 06:56:46.866784 (+ 12378us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.866791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.867516 (+ 725us) server_negotiation.cc:407] Connection header received
0812 06:56:46.867742 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.867751 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.867850 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.867991 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:46.869913 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.871176 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.873204 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.873902 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.874042 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.884248 (+ 10206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34890: BlockingRecv error: recv got EOF from 127.0.0.1:34890 (error 108)
Metrics: {"server-negotiator.queue_time_us":12209,"thread_start_us":12407,"threads_started":1}
Aug 12 06:56:46 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM
I20260812 06:56:46.905565 8623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.886280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34894 (local address 127.26.226.62:36507)
0812 06:56:46.886452 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.886458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.886998 (+ 540us) server_negotiation.cc:407] Connection header received
0812 06:56:46.887172 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.887178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.887278 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.887400 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:46.889339 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.890444 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.892367 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.893032 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.893183 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.898401 (+ 5218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:46.898407 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:46.898413 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:46.898459 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:46.902101 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.902947 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.902953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.902957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.903104 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.903628 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.903634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.903638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.904037 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:46.904367 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.904939 (+ 572us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.905180 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:56:46.910585 7578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:46.911399 7578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:46.919922 7830 tablet_service.cc:1467] Processing CreateTablet for tablet 3cc264b4e8794a6088dc67f561f484df (DEFAULT_TABLE table=test-table [id=badc3aca08594c62bfcf376ef96ff090]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:46.920130 8302 tablet_service.cc:1467] Processing CreateTablet for tablet 3cc264b4e8794a6088dc67f561f484df (DEFAULT_TABLE table=test-table [id=badc3aca08594c62bfcf376ef96ff090]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:46.920553 7830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cc264b4e8794a6088dc67f561f484df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:46.920605 8302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cc264b4e8794a6088dc67f561f484df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:46.924744 8058 tablet_service.cc:1467] Processing CreateTablet for tablet 3cc264b4e8794a6088dc67f561f484df (DEFAULT_TABLE table=test-table [id=badc3aca08594c62bfcf376ef96ff090]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:46.925184 8058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cc264b4e8794a6088dc67f561f484df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:46.952950 8637 tablet_bootstrap.cc:492] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242: Bootstrap starting.
I20260812 06:56:46.956104 8633 tablet_bootstrap.cc:492] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb: Bootstrap starting.
I20260812 06:56:46.957136 8637 tablet_bootstrap.cc:654] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:46.960775 8633 tablet_bootstrap.cc:654] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:46.964474 8637 tablet_bootstrap.cc:492] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242: No bootstrap required, opened a new log
I20260812 06:56:46.964736 8637 ts_tablet_manager.cc:1397] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260812 06:56:46.965586 8637 raft_consensus.cc:348] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.965966 8637 raft_consensus.cc:374] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:46.966637 8631 tablet_bootstrap.cc:492] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e: Bootstrap starting.
I20260812 06:56:46.966791 8637 raft_consensus.cc:729] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933197931f264cf1b228fdcdb0229242, State: Initialized, Role: FOLLOWER
I20260812 06:56:46.967140 8637 consensus_queue.cc:260] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.968384 8637 ts_tablet_manager.cc:1428] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:56:46.971604 8631 tablet_bootstrap.cc:654] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:46.975163 8633 tablet_bootstrap.cc:492] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb: No bootstrap required, opened a new log
I20260812 06:56:46.975339 8633 ts_tablet_manager.cc:1397] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.002s
I20260812 06:56:46.976279 8633 raft_consensus.cc:348] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.976589 8633 raft_consensus.cc:374] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:46.976699 8633 raft_consensus.cc:729] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58767d36b6704b18bac8a98076f623cb, State: Initialized, Role: FOLLOWER
I20260812 06:56:46.976999 8633 consensus_queue.cc:260] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.977802 8633 ts_tablet_manager.cc:1428] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:46.978298 8631 tablet_bootstrap.cc:492] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e: No bootstrap required, opened a new log
I20260812 06:56:46.978448 8631 ts_tablet_manager.cc:1397] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260812 06:56:46.979444 8631 raft_consensus.cc:348] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.979708 8631 raft_consensus.cc:374] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:46.979811 8631 raft_consensus.cc:729] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7436a98362ff4489a0255322b0a0753e, State: Initialized, Role: FOLLOWER
I20260812 06:56:46.980114 8631 consensus_queue.cc:260] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:46.983609 8631 ts_tablet_manager.cc:1428] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260812 06:56:47.104805 8553 raft_consensus.cc:482] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:47.105073 8553 raft_consensus.cc:504] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.105677 8553 leader_election.cc:288] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997), 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031)
I20260812 06:56:47.106477 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc264b4e8794a6088dc67f561f484df" candidate_uuid: "7436a98362ff4489a0255322b0a0753e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933197931f264cf1b228fdcdb0229242" is_pre_election: true
I20260812 06:56:47.106503 8343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc264b4e8794a6088dc67f561f484df" candidate_uuid: "7436a98362ff4489a0255322b0a0753e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58767d36b6704b18bac8a98076f623cb" is_pre_election: true
I20260812 06:56:47.106779 8093 raft_consensus.cc:2454] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7436a98362ff4489a0255322b0a0753e in term 0.
I20260812 06:56:47.106822 8343 raft_consensus.cc:2454] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7436a98362ff4489a0255322b0a0753e in term 0.
I20260812 06:56:47.107303 7742 leader_election.cc:302] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7436a98362ff4489a0255322b0a0753e, 933197931f264cf1b228fdcdb0229242; no voters:
I20260812 06:56:47.107698 8553 raft_consensus.cc:2781] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:47.107823 8553 raft_consensus.cc:482] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:47.107940 8553 raft_consensus.cc:3037] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:47.110046 8553 raft_consensus.cc:504] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.110599 8553 leader_election.cc:288] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [CANDIDATE]: Term 1 election: Requested vote from peers 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997), 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031)
I20260812 06:56:47.111174 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc264b4e8794a6088dc67f561f484df" candidate_uuid: "7436a98362ff4489a0255322b0a0753e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933197931f264cf1b228fdcdb0229242"
I20260812 06:56:47.111241 8343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc264b4e8794a6088dc67f561f484df" candidate_uuid: "7436a98362ff4489a0255322b0a0753e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58767d36b6704b18bac8a98076f623cb"
I20260812 06:56:47.111402 8093 raft_consensus.cc:3037] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:47.111487 8343 raft_consensus.cc:3037] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:47.113742 8093 raft_consensus.cc:2454] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7436a98362ff4489a0255322b0a0753e in term 1.
I20260812 06:56:47.114176 7742 leader_election.cc:302] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7436a98362ff4489a0255322b0a0753e, 933197931f264cf1b228fdcdb0229242; no voters:
I20260812 06:56:47.114475 8553 raft_consensus.cc:2781] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:47.114682 8553 raft_consensus.cc:686] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 1 LEADER]: Becoming Leader. State: Replica: 7436a98362ff4489a0255322b0a0753e, State: Running, Role: LEADER
I20260812 06:56:47.114954 8553 consensus_queue.cc:237] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.115079 8343 raft_consensus.cc:2454] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7436a98362ff4489a0255322b0a0753e in term 1.
I20260812 06:56:47.118121 7581 catalog_manager.cc:5495] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7436a98362ff4489a0255322b0a0753e (127.26.226.1). New cstate: current_term: 1 leader_uuid: "7436a98362ff4489a0255322b0a0753e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } health_report { overall_health: UNKNOWN } } }
I20260812 06:56:47.135949 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.129400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.26.226.1:40059)
0812 06:56:47.129610 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.129618 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.129816 (+ 198us) server_negotiation.cc:407] Connection header received
0812 06:56:47.130017 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.130022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.130125 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.130241 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:47.130662 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.131815 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.133788 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.134437 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.134598 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.135116 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:47.135263 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:47.135371 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.135551 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.135645 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:56:47.140023 8343 raft_consensus.cc:1261] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Refusing update from remote peer 7436a98362ff4489a0255322b0a0753e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:47.140022 8093 raft_consensus.cc:1261] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Refusing update from remote peer 7436a98362ff4489a0255322b0a0753e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:47.140834 8479 consensus_queue.cc:1035] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [LEADER]: Connected to new peer: Peer: permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:47.141168 8578 consensus_queue.cc:1035] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:47.173231 7581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:47.173600 7581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:47.177309 7581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837: Sending DeleteTablet for 3 replicas of tablet 3cc264b4e8794a6088dc67f561f484df
I20260812 06:56:47.178381 8058 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cc264b4e8794a6088dc67f561f484df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:47 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:56736
I20260812 06:56:47.178434 7830 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cc264b4e8794a6088dc67f561f484df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:47 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:35658
I20260812 06:56:47.178737 8597 tablet_replica.cc:330] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e: stopping tablet replica
I20260812 06:56:47.178771 8600 tablet_replica.cc:330] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242: stopping tablet replica
I20260812 06:56:47.178987 8600 raft_consensus.cc:2229] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:47.178977 8597 raft_consensus.cc:2229] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:47.179276 8600 raft_consensus.cc:2258] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:47.179565 8597 raft_consensus.cc:2258] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:47.180295 8302 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cc264b4e8794a6088dc67f561f484df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:47 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:56994
I20260812 06:56:47.180612 8602 tablet_replica.cc:330] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb: stopping tablet replica
I20260812 06:56:47.180848 8602 raft_consensus.cc:2229] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:47.181107 8602 raft_consensus.cc:2258] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:47.181655 7578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:47.182381 7578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:47.184633 8602 ts_tablet_manager.cc:1905] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:47.189224 8597 ts_tablet_manager.cc:1905] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:47.191440 7830 tablet_service.cc:1467] Processing CreateTablet for tablet 8c702f09a58e45faa449715bb6241026 (DEFAULT_TABLE table=test-table [id=38418f943a514350b4e819e0bf5fac2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:47.191952 7830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c702f09a58e45faa449715bb6241026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:47.191905 8058 tablet_service.cc:1467] Processing CreateTablet for tablet 8c702f09a58e45faa449715bb6241026 (DEFAULT_TABLE table=test-table [id=38418f943a514350b4e819e0bf5fac2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:47.192394 8058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c702f09a58e45faa449715bb6241026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:47.196825 8302 tablet_service.cc:1467] Processing CreateTablet for tablet 8c702f09a58e45faa449715bb6241026 (DEFAULT_TABLE table=test-table [id=38418f943a514350b4e819e0bf5fac2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:47.197355 8302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c702f09a58e45faa449715bb6241026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:47.203385 8637 tablet_bootstrap.cc:492] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242: Bootstrap starting.
I20260812 06:56:47.205219 8637 tablet_bootstrap.cc:654] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:47.207863 8637 tablet_bootstrap.cc:492] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242: No bootstrap required, opened a new log
I20260812 06:56:47.208101 8637 ts_tablet_manager.cc:1397] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260812 06:56:47.208962 8637 raft_consensus.cc:348] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.209246 8637 raft_consensus.cc:374] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:47.209333 8637 raft_consensus.cc:729] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933197931f264cf1b228fdcdb0229242, State: Initialized, Role: FOLLOWER
I20260812 06:56:47.209631 8637 consensus_queue.cc:260] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.210358 8637 ts_tablet_manager.cc:1428] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 06:56:47.217231 8633 tablet_bootstrap.cc:492] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb: Bootstrap starting.
I20260812 06:56:47.218498 8597 ts_tablet_manager.cc:1918] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:47.218701 8597 log.cc:1196] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/3cc264b4e8794a6088dc67f561f484df
I20260812 06:56:47.219110 8597 ts_tablet_manager.cc:1939] T 3cc264b4e8794a6088dc67f561f484df P 7436a98362ff4489a0255322b0a0753e: Deleting consensus metadata
I20260812 06:56:47.219233 8633 tablet_bootstrap.cc:654] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:47.220510 7555 catalog_manager.cc:4841] TS 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059): tablet 3cc264b4e8794a6088dc67f561f484df (table test-table [id=badc3aca08594c62bfcf376ef96ff090]) successfully deleted
I20260812 06:56:47.221246 8631 tablet_bootstrap.cc:492] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e: Bootstrap starting.
I20260812 06:56:47.222085 8602 ts_tablet_manager.cc:1918] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:47.222268 8602 log.cc:1196] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/3cc264b4e8794a6088dc67f561f484df
I20260812 06:56:47.222270 8600 ts_tablet_manager.cc:1905] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:47.222779 8602 ts_tablet_manager.cc:1939] T 3cc264b4e8794a6088dc67f561f484df P 58767d36b6704b18bac8a98076f623cb: Deleting consensus metadata
I20260812 06:56:47.223006 8631 tablet_bootstrap.cc:654] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:47.228286 7553 catalog_manager.cc:4841] TS 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031): tablet 3cc264b4e8794a6088dc67f561f484df (table test-table [id=badc3aca08594c62bfcf376ef96ff090]) successfully deleted
I20260812 06:56:47.231679 8600 ts_tablet_manager.cc:1918] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:47.231881 8600 log.cc:1196] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/3cc264b4e8794a6088dc67f561f484df
I20260812 06:56:47.232326 8600 ts_tablet_manager.cc:1939] T 3cc264b4e8794a6088dc67f561f484df P 933197931f264cf1b228fdcdb0229242: Deleting consensus metadata
I20260812 06:56:47.233507 8633 tablet_bootstrap.cc:492] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb: No bootstrap required, opened a new log
I20260812 06:56:47.233492 7553 catalog_manager.cc:4841] TS 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997): tablet 3cc264b4e8794a6088dc67f561f484df (table test-table [id=badc3aca08594c62bfcf376ef96ff090]) successfully deleted
I20260812 06:56:47.233820 8633 ts_tablet_manager.cc:1397] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.001s
I20260812 06:56:47.234906 8633 raft_consensus.cc:348] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.235188 8633 raft_consensus.cc:374] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:47.235328 8633 raft_consensus.cc:729] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58767d36b6704b18bac8a98076f623cb, State: Initialized, Role: FOLLOWER
I20260812 06:56:47.235611 8633 consensus_queue.cc:260] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.236217 8633 ts_tablet_manager.cc:1428] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:56:47.240407 8631 tablet_bootstrap.cc:492] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e: No bootstrap required, opened a new log
I20260812 06:56:47.240600 8631 ts_tablet_manager.cc:1397] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.008s
I20260812 06:56:47.241460 8631 raft_consensus.cc:348] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.241750 8631 raft_consensus.cc:374] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:47.241851 8631 raft_consensus.cc:729] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7436a98362ff4489a0255322b0a0753e, State: Initialized, Role: FOLLOWER
I20260812 06:56:47.242118 8631 consensus_queue.cc:260] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.242713 8631 ts_tablet_manager.cc:1428] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 06:56:47.257051 8489 raft_consensus.cc:482] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:47.257301 8489 raft_consensus.cc:504] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.258003 8489 leader_election.cc:288] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059), 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031)
I20260812 06:56:47.258652 7859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c702f09a58e45faa449715bb6241026" candidate_uuid: "933197931f264cf1b228fdcdb0229242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7436a98362ff4489a0255322b0a0753e" is_pre_election: true
I20260812 06:56:47.258798 8346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c702f09a58e45faa449715bb6241026" candidate_uuid: "933197931f264cf1b228fdcdb0229242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58767d36b6704b18bac8a98076f623cb" is_pre_election: true
I20260812 06:56:47.258958 7859 raft_consensus.cc:2454] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 933197931f264cf1b228fdcdb0229242 in term 0.
I20260812 06:56:47.259140 8346 raft_consensus.cc:2454] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 933197931f264cf1b228fdcdb0229242 in term 0.
I20260812 06:56:47.259502 7981 leader_election.cc:302] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7436a98362ff4489a0255322b0a0753e, 933197931f264cf1b228fdcdb0229242; no voters:
I20260812 06:56:47.259948 8489 raft_consensus.cc:2781] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:47.260075 8489 raft_consensus.cc:482] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:47.260205 8489 raft_consensus.cc:3037] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:47.263453 8489 raft_consensus.cc:504] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.264820 7859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c702f09a58e45faa449715bb6241026" candidate_uuid: "933197931f264cf1b228fdcdb0229242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7436a98362ff4489a0255322b0a0753e"
I20260812 06:56:47.264847 8346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c702f09a58e45faa449715bb6241026" candidate_uuid: "933197931f264cf1b228fdcdb0229242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58767d36b6704b18bac8a98076f623cb"
I20260812 06:56:47.265093 8346 raft_consensus.cc:3037] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:47.265094 7859 raft_consensus.cc:3037] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:47.268461 7859 raft_consensus.cc:2454] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 933197931f264cf1b228fdcdb0229242 in term 1.
I20260812 06:56:47.268975 7981 leader_election.cc:302] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7436a98362ff4489a0255322b0a0753e, 933197931f264cf1b228fdcdb0229242; no voters:
I20260812 06:56:47.269824 8489 leader_election.cc:288] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [CANDIDATE]: Term 1 election: Requested vote from peers 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059), 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031)
I20260812 06:56:47.270017 8489 raft_consensus.cc:2781] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:47.270179 8489 raft_consensus.cc:686] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 1 LEADER]: Becoming Leader. State: Replica: 933197931f264cf1b228fdcdb0229242, State: Running, Role: LEADER
I20260812 06:56:47.270316 8346 raft_consensus.cc:2454] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 933197931f264cf1b228fdcdb0229242 in term 1.
I20260812 06:56:47.270466 8489 consensus_queue.cc:237] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } }
I20260812 06:56:47.291425 7578 catalog_manager.cc:5495] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 reported cstate change: term changed from 0 to 1, leader changed from <none> to 933197931f264cf1b228fdcdb0229242 (127.26.226.2). New cstate: current_term: 1 leader_uuid: "933197931f264cf1b228fdcdb0229242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "933197931f264cf1b228fdcdb0229242" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 34997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 } health_report { overall_health: UNKNOWN } } }
I20260812 06:56:47.335271 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.324503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.26.226.2:34997)
0812 06:56:47.329034 (+ 4531us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.329041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.329065 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:56:47.329134 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.329140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.329223 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.329330 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:47.329833 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.330991 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.333060 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.333724 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.333845 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.334376 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:47.334512 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:47.334608 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.334795 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.334876 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4387,"thread_start_us":4276,"threads_started":1}
I20260812 06:56:47.354076 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.346565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:40059 (local address 127.26.226.2:50527)
0812 06:56:47.348444 (+ 1879us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:47.348464 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:47.348613 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:47.349029 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:47.349035 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:47.349048 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:56:47.349338 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.349348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.350846 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.350850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.351891 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.351902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.352987 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.352992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.353126 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.353181 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:56:47.353620 (+ 439us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:56:47.353628 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 06:56:47.353691 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 06:56:47.353789 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1707,"thread_start_us":1534,"threads_started":1}
I20260812 06:56:47.354171 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.346593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:50527 (local address 127.26.226.1:40059)
0812 06:56:47.346763 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.346769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.348555 (+ 1786us) server_negotiation.cc:407] Connection header received
0812 06:56:47.348757 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.348762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.348854 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.348964 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:47.349511 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.350702 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.352034 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.352760 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.352900 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.353363 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:47.353515 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:47.353611 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.353743 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.353825 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:56:47.362987 8346 raft_consensus.cc:1261] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Refusing update from remote peer 933197931f264cf1b228fdcdb0229242: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:47.363610 8665 consensus_queue.cc:1035] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58767d36b6704b18bac8a98076f623cb" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:47.363945 7859 raft_consensus.cc:1261] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Refusing update from remote peer 933197931f264cf1b228fdcdb0229242: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:47.364540 8665 consensus_queue.cc:1035] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7436a98362ff4489a0255322b0a0753e" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:47.398664 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.391982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:46903 (local address 127.26.226.2:34997)
0812 06:56:47.392555 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.392562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.393051 (+ 489us) server_negotiation.cc:407] Connection header received
0812 06:56:47.393316 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.393319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.393382 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.393457 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:47.394027 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.394856 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.396777 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.397435 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.397564 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.397932 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:47.398068 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:47.398163 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.398298 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.398383 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435}
I20260812 06:56:47.398712 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.391676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:34997 (local address 127.26.226.1:46903)
0812 06:56:47.392926 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:47.392947 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:47.393155 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:47.393500 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:47.393505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:47.393518 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:56:47.393840 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.393852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.395047 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.395052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.396624 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.396641 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.397599 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.397604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.397740 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.397777 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:56:47.398175 (+ 398us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:56:47.398185 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 06:56:47.398253 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 06:56:47.398353 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":766,"threads_started":1}
I20260812 06:56:47.440299 7581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:47.440582 7581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:47.443493 7581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4cf7aedca9cd4ac4a3696cbf3bdd0837: Sending DeleteTablet for 3 replicas of tablet 8c702f09a58e45faa449715bb6241026
I20260812 06:56:47.444489 8302 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c702f09a58e45faa449715bb6241026 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:47 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:56994
I20260812 06:56:47.444710 8058 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c702f09a58e45faa449715bb6241026 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:47 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:56736
I20260812 06:56:47.445026 8602 tablet_replica.cc:330] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb: stopping tablet replica
I20260812 06:56:47.445065 8600 tablet_replica.cc:330] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242: stopping tablet replica
I20260812 06:56:47.445228 8600 raft_consensus.cc:2229] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:47.445215 8602 raft_consensus.cc:2229] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:47.445415 8602 raft_consensus.cc:2258] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:47.445475 8600 raft_consensus.cc:2258] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:47.445762 7830 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c702f09a58e45faa449715bb6241026 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:47 UTC) from {username='oryx', principal='oryx/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:35658
I20260812 06:56:47.446081 8597 tablet_replica.cc:330] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e: stopping tablet replica
I20260812 06:56:47.446101 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 7710
I20260812 06:56:47.446260 8597 raft_consensus.cc:2229] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:47.446491 8597 raft_consensus.cc:2258] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:47.446967 8602 ts_tablet_manager.cc:1905] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:47.449157 8597 ts_tablet_manager.cc:1905] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:47.452886 8600 ts_tablet_manager.cc:1905] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:47.453657 8602 ts_tablet_manager.cc:1918] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:56:47.453790 8602 log.cc:1196] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/8c702f09a58e45faa449715bb6241026
I20260812 06:56:47.454254 8602 ts_tablet_manager.cc:1939] T 8c702f09a58e45faa449715bb6241026 P 58767d36b6704b18bac8a98076f623cb: Deleting consensus metadata
I20260812 06:56:47.456578 7553 catalog_manager.cc:4841] TS 58767d36b6704b18bac8a98076f623cb (127.26.226.3:41031): tablet 8c702f09a58e45faa449715bb6241026 (table test-table [id=38418f943a514350b4e819e0bf5fac2f]) successfully deleted
I20260812 06:56:47.459383 8600 ts_tablet_manager.cc:1918] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:56:47.459522 8600 log.cc:1196] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/8c702f09a58e45faa449715bb6241026
I20260812 06:56:47.460104 8600 ts_tablet_manager.cc:1939] T 8c702f09a58e45faa449715bb6241026 P 933197931f264cf1b228fdcdb0229242: Deleting consensus metadata
I20260812 06:56:47.461280 7553 catalog_manager.cc:4841] TS 933197931f264cf1b228fdcdb0229242 (127.26.226.2:34997): tablet 8c702f09a58e45faa449715bb6241026 (table test-table [id=38418f943a514350b4e819e0bf5fac2f]) successfully deleted
I20260812 06:56:47.461477 8597 ts_tablet_manager.cc:1918] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:56:47.461649 8597 log.cc:1196] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/8c702f09a58e45faa449715bb6241026
I20260812 06:56:47.462057 8597 ts_tablet_manager.cc:1939] T 8c702f09a58e45faa449715bb6241026 P 7436a98362ff4489a0255322b0a0753e: Deleting consensus metadata
I20260812 06:56:47.462986 7555 catalog_manager.cc:4841] TS 7436a98362ff4489a0255322b0a0753e (127.26.226.1:40059): tablet 8c702f09a58e45faa449715bb6241026 (table test-table [id=38418f943a514350b4e819e0bf5fac2f]) successfully deleted
Aug 12 06:56:47 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.1@KRBTEST.COM
I20260812 06:56:47.475462 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.449561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.26.226.1:40059)
0812 06:56:47.449856 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.449866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.457220 (+ 7354us) server_negotiation.cc:407] Connection header received
0812 06:56:47.457660 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.457668 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.457747 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.457818 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:47.459910 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.461091 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.463192 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.463738 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.463869 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.468762 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:47.468778 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:47.468796 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:47.468841 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:47.471980 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:47.472852 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:47.472862 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:47.472864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:47.473074 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:47.473495 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:47.473500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:47.473504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:47.474041 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:47.474359 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.474695 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.474917 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260812 06:56:47.476260 7930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:47.624908 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 7710
I20260812 06:56:47.663861 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 7943
Aug 12 06:56:47 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.2@KRBTEST.COM
I20260812 06:56:47.679914 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.664724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56766 (local address 127.26.226.2:34997)
0812 06:56:47.664936 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.664942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.665043 (+ 101us) server_negotiation.cc:407] Connection header received
0812 06:56:47.665272 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.665276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.665362 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.665475 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:47.666746 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.667520 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.668903 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.669357 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.669510 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.673786 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:47.673804 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:47.673820 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:47.673855 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:47.676937 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:47.677677 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:47.677684 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:47.677686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:47.677859 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:47.678355 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:47.678358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:47.678361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:47.678757 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:47.678990 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.679456 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.679643 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:56:47.680701 8170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:47.810686 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 7943
I20260812 06:56:47.845419 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 8185
Aug 12 06:56:47 dist-test-slave-t3q3 krb5kdc[7515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.226.3@KRBTEST.COM
I20260812 06:56:47.860848 8689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.846238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.26.226.3:41031)
0812 06:56:47.846740 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.846744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.846760 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:56:47.846818 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.846823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.846892 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.846987 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:47.848341 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.849025 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.850219 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.850588 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.850680 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.855120 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:47.855137 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:47.855154 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:47.855185 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:47.857968 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:47.858576 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:47.858584 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:47.858586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:47.858771 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:47.859123 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:47.859127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:47.859129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:47.859538 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:47.859772 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.860322 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.860590 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":270,"threads_started":1}
I20260812 06:56:47.861529 8412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:47.986950 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 8185
I20260812 06:56:48.016242 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 7534
I20260812 06:56:48.017468 7655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:48.135536 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 7534
2026-08-12T06:56:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7767 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:48 dist-test-slave-t3q3 krb5kdc[8699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:48 dist-test-slave-t3q3 krb5kdc[8699](info): set up 2 sockets
Aug 12 06:56:48 dist-test-slave-t3q3 krb5kdc[8699](info): commencing operation
krb5kdc: starting...
W20260812 06:56:49.505911 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:56:49 dist-test-slave-t3q3 krb5kdc[8699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:56:49.800341 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:46175
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:37803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:46175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:50.151945 8728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:50.152359 8728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:50.152544 8728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:50.159754 8728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:50.159919 8728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:50.160039 8728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:50.160163 8728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:50.160249 8728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:50.170197 8728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:37803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:46175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:46175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:50.171581 8728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:50.172928 8728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:50.185458 8737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:50.185434 8739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:50.185544 8743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:50.186199 8728 server_base.cc:1034] running on GCE node
I20260812 06:56:50.187041 8728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:50.188462 8728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:50.189700 8728 hybrid_clock.cc:648] HybridClock initialized: now 1786517810189650 us; error 59 us; skew 500 ppm
I20260812 06:56:50.190053 8728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T06:56:50Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2121 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:50 dist-test-slave-t3q3 krb5kdc[8753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:50 dist-test-slave-t3q3 krb5kdc[8753](info): set up 2 sockets
Aug 12 06:56:50 dist-test-slave-t3q3 krb5kdc[8753](info): commencing operation
krb5kdc: starting...
W20260812 06:56:51.630877 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:56:51 dist-test-slave-t3q3 krb5kdc[8753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:56:51.935184 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34153
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:34153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:52.286753 8773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:52.287031 8773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:52.287227 8773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:52.294541 8773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:52.294641 8773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:52.294770 8773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:52.294847 8773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:52.294905 8773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:52.304746 8773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:34153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:52.306100 8773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:52.307410 8773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:52.319407 8781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:52.319516 8782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:52.319516 8786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:52.320470 8773 server_base.cc:1034] running on GCE node
I20260812 06:56:52.321322 8773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:52.322772 8773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:52.324007 8773 hybrid_clock.cc:648] HybridClock initialized: now 1786517812323970 us; error 49 us; skew 500 ppm
I20260812 06:56:52.324415 8773 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:52 dist-test-slave-t3q3 krb5kdc[8753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517812, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:52.332213 8773 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:56:52.333889 8773 webserver.cc:460] Webserver started at http://127.26.226.62:45517/ using document root <none> and password file <none>
I20260812 06:56:52.334590 8773 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:52.334697 8773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:52.335116 8773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:52.338685 8773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "c365ab48199f456a9e8b4cd411fafe8f"
format_stamp: "Formatted at 2026-08-12 06:56:52 on dist-test-slave-t3q3"
server_key: "df73fa87cec11b1e71dc8dbca9faf8d8"
server_key_iv: "9b8705de5ef3aa9a17b452c7b04401bd"
server_key_version: "encryptionkey@0"
I20260812 06:56:52.339510 8773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "c365ab48199f456a9e8b4cd411fafe8f"
format_stamp: "Formatted at 2026-08-12 06:56:52 on dist-test-slave-t3q3"
server_key: "df73fa87cec11b1e71dc8dbca9faf8d8"
server_key_iv: "9b8705de5ef3aa9a17b452c7b04401bd"
server_key_version: "encryptionkey@0"
I20260812 06:56:52.345142 8773 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260812 06:56:52.348878 8804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:52.350308 8773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:52.350517 8773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "c365ab48199f456a9e8b4cd411fafe8f"
format_stamp: "Formatted at 2026-08-12 06:56:52 on dist-test-slave-t3q3"
server_key: "df73fa87cec11b1e71dc8dbca9faf8d8"
server_key_iv: "9b8705de5ef3aa9a17b452c7b04401bd"
server_key_version: "encryptionkey@0"
I20260812 06:56:52.350744 8773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:52.373250 8773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:52.378082 8773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:52.378429 8773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:52.397533 8773 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:34153
I20260812 06:56:52.397562 8889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:34153 every 8 connection(s)
I20260812 06:56:52.399000 8773 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:52.402827 8892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:52.404894 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 8773
I20260812 06:56:52.405133 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:56:52.405825 27528 external_mini_cluster.cc:1428] Setting key f559d0ade4eb31345bf6a79683d0d2f2
I20260812 06:56:52.415577 8892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f: Bootstrap starting.
I20260812 06:56:52.419662 8892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:52.421093 8892 log.cc:824] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:56:52 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:52.424044 8892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f: No bootstrap required, opened a new log
I20260812 06:56:52.430001 8896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:52.407939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.26.226.62:34153)
0812 06:56:52.408903 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:52.408915 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:52.408958 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:56:52.410859 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:52.410883 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:52.411306 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:52.411729 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:52.413681 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:52.414997 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:52.416793 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:52.417486 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:52.417627 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:52.422524 (+ 4897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:52.422533 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:52.422548 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:52.422584 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:52.425763 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:52.426547 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:52.426554 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:52.426560 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:52.426739 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:52.427160 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:52.427164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:52.427166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:52.427718 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:52.428047 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:52.428546 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 06:56:52.428966 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":449,"threads_started":1}
I20260812 06:56:52.432255 8892 raft_consensus.cc:348] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } }
I20260812 06:56:52.432646 8892 raft_consensus.cc:374] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:52.432783 8892 raft_consensus.cc:729] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c365ab48199f456a9e8b4cd411fafe8f, State: Initialized, Role: FOLLOWER
I20260812 06:56:52.433470 8892 consensus_queue.cc:260] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } }
I20260812 06:56:52.433740 8892 raft_consensus.cc:388] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:52.433893 8892 raft_consensus.cc:482] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:52.434075 8892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:52.436720 8892 raft_consensus.cc:504] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } }
I20260812 06:56:52.437250 8892 leader_election.cc:302] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c365ab48199f456a9e8b4cd411fafe8f; no voters:
I20260812 06:56:52.437858 8892 leader_election.cc:288] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:52.438006 8898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:52.438742 8898 raft_consensus.cc:686] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 1 LEADER]: Becoming Leader. State: Replica: c365ab48199f456a9e8b4cd411fafe8f, State: Running, Role: LEADER
I20260812 06:56:52.439379 8898 consensus_queue.cc:237] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } }
I20260812 06:56:52.440188 8892 sys_catalog.cc:564] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:52.442835 8909 sys_catalog.cc:455] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c365ab48199f456a9e8b4cd411fafe8f. Latest consensus state: current_term: 1 leader_uuid: "c365ab48199f456a9e8b4cd411fafe8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } } }
I20260812 06:56:52.443321 8909 sys_catalog.cc:458] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:52.442862 8900 sys_catalog.cc:455] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c365ab48199f456a9e8b4cd411fafe8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } } }
I20260812 06:56:52.444290 8900 sys_catalog.cc:458] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:52.449120 8916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:52.454586 8916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:52.466614 8916 catalog_manager.cc:1347] Generated new cluster ID: 303a71f91de344729e73594f3485d70d
I20260812 06:56:52.466766 8916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:52.486475 8916 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:52.488505 8916 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:52.500625 8916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f: Generated new TSK 0
I20260812 06:56:52.501596 8916 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:56:52.581817 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34153
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:52.943033 8947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:52.943331 8947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:52.943539 8947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:52.950980 8947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:52.951128 8947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:52.951340 8947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:52.961694 8947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:52.963220 8947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:52.964636 8947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:52.977957 8958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:52.977980 8961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:52.977903 8963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:52.979210 8947 server_base.cc:1034] running on GCE node
I20260812 06:56:52.979782 8947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:52.980861 8947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:52.982103 8947 hybrid_clock.cc:648] HybridClock initialized: now 1786517812982064 us; error 35 us; skew 500 ppm
I20260812 06:56:52.982451 8947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:52 dist-test-slave-t3q3 krb5kdc[8753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517812, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:52.985941 8947 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:56:52.987586 8947 webserver.cc:460] Webserver started at http://127.26.226.1:46517/ using document root <none> and password file <none>
I20260812 06:56:52.988358 8947 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:52.988472 8947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:52.988755 8947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:52.991801 8947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "ea7e6681f8094a0688f09d864d9b0349"
format_stamp: "Formatted at 2026-08-12 06:56:52 on dist-test-slave-t3q3"
server_key: "a51f6c2eab7398557dbbf5c2bf68587f"
server_key_iv: "ffdef2110876c0130995535746a1a69f"
server_key_version: "encryptionkey@0"
I20260812 06:56:52.992682 8947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "ea7e6681f8094a0688f09d864d9b0349"
format_stamp: "Formatted at 2026-08-12 06:56:52 on dist-test-slave-t3q3"
server_key: "a51f6c2eab7398557dbbf5c2bf68587f"
server_key_iv: "ffdef2110876c0130995535746a1a69f"
server_key_version: "encryptionkey@0"
I20260812 06:56:52.997943 8947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:56:53.001786 8974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:53.003347 8947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:53.003543 8947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "ea7e6681f8094a0688f09d864d9b0349"
format_stamp: "Formatted at 2026-08-12 06:56:52 on dist-test-slave-t3q3"
server_key: "a51f6c2eab7398557dbbf5c2bf68587f"
server_key_iv: "ffdef2110876c0130995535746a1a69f"
server_key_version: "encryptionkey@0"
I20260812 06:56:53.003758 8947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:53.027143 8947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:53.031812 8947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:53.032148 8947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:53.033296 8947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:53.034672 8947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:53.034787 8947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:53.034942 8947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:53.035025 8947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:53.070688 8947 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:34859
I20260812 06:56:53.070778 9172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:34859 every 8 connection(s)
I20260812 06:56:53.072118 8947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:53.082175 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 8947
I20260812 06:56:53.082392 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:53.083060 27528 external_mini_cluster.cc:1428] Setting key 8f3546048159b27f5791dfe895427255
Aug 12 06:56:53 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517812, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:53.092413 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:53.074726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:54581 (local address 127.26.226.62:34153)
0812 06:56:53.075204 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:53.075209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:53.076018 (+ 809us) server_negotiation.cc:407] Connection header received
0812 06:56:53.077928 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:53.077934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:53.078026 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:53.078150 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:53.079650 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.080453 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.082026 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.082535 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.082690 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:53.086586 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:53.086592 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:53.086595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:53.086627 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:53.089267 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:53.090011 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:53.090018 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:53.090022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:53.090151 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:53.090602 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:53.090608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:53.090611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:53.090882 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:53.091044 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:53.091764 (+ 720us) server_negotiation.cc:299] Negotiation successful
0812 06:56:53.092002 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":263,"threads_started":1}
I20260812 06:56:53.093372 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:53.075143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:54581)
0812 06:56:53.075844 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:53.075882 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:53.077654 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:53.078400 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:53.078407 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:53.078916 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:53.079466 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:53.079483 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.080629 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.080633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:53.081857 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:53.081870 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.082764 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.082773 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:53.083025 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:53.083819 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:53.083852 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:53.086356 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:53.089425 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:53.089435 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:53.089450 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:53.089883 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:53.090276 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:53.090281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:53.090285 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:53.090481 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:53.091019 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:53.091027 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:53.091375 (+ 348us) client_negotiation.cc:769] Sending connection context
0812 06:56:53.091661 (+ 286us) client_negotiation.cc:240] Negotiation successful
0812 06:56:53.092217 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":270,"threads_started":1}
I20260812 06:56:53.095645 9173 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34153
I20260812 06:56:53.095968 9173 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:53.096836 9173 heartbeater.cc:511] Master 127.26.226.62:34153 requested a full tablet report, sending...
I20260812 06:56:53.099197 8829 ts_manager.cc:194] Registered new tserver with Master: ea7e6681f8094a0688f09d864d9b0349 (127.26.226.1:34859)
I20260812 06:56:53.100705 8829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:54581
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:56:53.147796 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34153
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:53.517830 9182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:53.518110 9182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:53.518262 9182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:53.525662 9182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:53.525777 9182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:53.525972 9182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:53.536005 9182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:53.537456 9182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:53.538825 9182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:53.552593 9194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:53.552767 9192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:53.552745 9196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:53.553843 9182 server_base.cc:1034] running on GCE node
I20260812 06:56:53.554551 9182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:53.555557 9182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:53.556768 9182 hybrid_clock.cc:648] HybridClock initialized: now 1786517813556703 us; error 53 us; skew 500 ppm
I20260812 06:56:53.557149 9182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:53 dist-test-slave-t3q3 krb5kdc[8753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:53.560504 9182 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:56:53.562140 9182 webserver.cc:460] Webserver started at http://127.26.226.2:37235/ using document root <none> and password file <none>
I20260812 06:56:53.562844 9182 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:53.562987 9182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:53.563328 9182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:53.566395 9182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "6f014368642b4fdf89e428bc56c338a6"
format_stamp: "Formatted at 2026-08-12 06:56:53 on dist-test-slave-t3q3"
server_key: "f2dfc8e9a3e5af4157c9a152f2094362"
server_key_iv: "46cceebc608fcc60b14b3a69dc7892d5"
server_key_version: "encryptionkey@0"
I20260812 06:56:53.567235 9182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "6f014368642b4fdf89e428bc56c338a6"
format_stamp: "Formatted at 2026-08-12 06:56:53 on dist-test-slave-t3q3"
server_key: "f2dfc8e9a3e5af4157c9a152f2094362"
server_key_iv: "46cceebc608fcc60b14b3a69dc7892d5"
server_key_version: "encryptionkey@0"
I20260812 06:56:53.572651 9182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 06:56:53.576210 9210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:53.577590 9182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:53.577778 9182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "6f014368642b4fdf89e428bc56c338a6"
format_stamp: "Formatted at 2026-08-12 06:56:53 on dist-test-slave-t3q3"
server_key: "f2dfc8e9a3e5af4157c9a152f2094362"
server_key_iv: "46cceebc608fcc60b14b3a69dc7892d5"
server_key_version: "encryptionkey@0"
I20260812 06:56:53.577984 9182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:53.596753 9182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:53.601186 9182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:53.601511 9182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:53.602535 9182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:53.604008 9182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:53.604146 9182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260812 06:56:53.604339 9182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:53.604394 9182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:53.640096 9182 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33337
I20260812 06:56:53.640161 9408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33337 every 8 connection(s)
I20260812 06:56:53.641607 9182 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:53.649015 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 9182
I20260812 06:56:53.649221 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:53.649811 27528 external_mini_cluster.cc:1428] Setting key d8f5e2c389cf856b7de38b78d8236948
Aug 12 06:56:53 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:53.666098 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:53.644476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:41093 (local address 127.26.226.62:34153)
0812 06:56:53.644994 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:53.644999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:53.645925 (+ 926us) server_negotiation.cc:407] Connection header received
0812 06:56:53.647883 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:53.647889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:53.647991 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:53.648151 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:53.650362 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.651240 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.653803 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.654369 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.654476 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:53.659206 (+ 4730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:53.659212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:53.659216 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:53.659250 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:53.662234 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:53.663457 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:53.663466 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:53.663475 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:53.663631 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:53.664145 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:53.664150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:53.664155 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:53.664453 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:53.664642 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:53.665424 (+ 782us) server_negotiation.cc:299] Negotiation successful
0812 06:56:53.665689 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":278,"threads_started":1}
I20260812 06:56:53.667161 9415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:53.644987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:41093)
0812 06:56:53.645739 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:53.645793 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:53.647577 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:53.648412 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:53.648421 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:53.649265 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:53.650136 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:53.650165 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.651433 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.651438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:53.653614 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:53.653636 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.654697 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.654708 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:53.655088 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:53.655958 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:53.656002 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:53.658895 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:53.662443 (+ 3548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:53.662455 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:53.662475 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:53.663259 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:53.663760 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:53.663765 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:53.663770 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:53.663982 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:53.664611 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:53.664620 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:53.665023 (+ 403us) client_negotiation.cc:769] Sending connection context
0812 06:56:53.665325 (+ 302us) client_negotiation.cc:240] Negotiation successful
0812 06:56:53.665924 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":292,"threads_started":1}
I20260812 06:56:53.669023 9411 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34153
I20260812 06:56:53.669395 9411 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:53.670415 9411 heartbeater.cc:511] Master 127.26.226.62:34153 requested a full tablet report, sending...
I20260812 06:56:53.672349 8829 ts_manager.cc:194] Registered new tserver with Master: 6f014368642b4fdf89e428bc56c338a6 (127.26.226.2:33337)
I20260812 06:56:53.673677 8829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:41093
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:56:53.719431 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34153
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:54.093408 9419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:54.093712 9419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:54.093873 9419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:54.101303 9419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:54.101410 9419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:54.101601 9419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:54.104532 9173 heartbeater.cc:503] Master 127.26.226.62:34153 was elected leader, sending a full tablet report...
I20260812 06:56:54.112835 9419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:54.114169 9419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:54.115504 9419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:54.129408 9435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:54.129468 9438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:54.129334 9440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:54.130797 9419 server_base.cc:1034] running on GCE node
I20260812 06:56:54.131493 9419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:54.132565 9419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:54.133836 9419 hybrid_clock.cc:648] HybridClock initialized: now 1786517814133771 us; error 59 us; skew 500 ppm
I20260812 06:56:54.134167 9419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:54 dist-test-slave-t3q3 krb5kdc[8753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517814, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:54.137919 9419 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:56:54.142409 9419 webserver.cc:460] Webserver started at http://127.26.226.3:40783/ using document root <none> and password file <none>
I20260812 06:56:54.143168 9419 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:54.143352 9419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:54.143707 9419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:54.146996 9419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "bc5e239f0ea644fb8cc8ba5a2a81c16b"
format_stamp: "Formatted at 2026-08-12 06:56:54 on dist-test-slave-t3q3"
server_key: "5326f3f1e0eb002bb14ce421d526f05c"
server_key_iv: "af2a3c935f8f46a55adcccec989af062"
server_key_version: "encryptionkey@0"
I20260812 06:56:54.148011 9419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "bc5e239f0ea644fb8cc8ba5a2a81c16b"
format_stamp: "Formatted at 2026-08-12 06:56:54 on dist-test-slave-t3q3"
server_key: "5326f3f1e0eb002bb14ce421d526f05c"
server_key_iv: "af2a3c935f8f46a55adcccec989af062"
server_key_version: "encryptionkey@0"
I20260812 06:56:54.153641 9419 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:56:54.157624 9449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:54.159001 9419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:54.159201 9419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "bc5e239f0ea644fb8cc8ba5a2a81c16b"
format_stamp: "Formatted at 2026-08-12 06:56:54 on dist-test-slave-t3q3"
server_key: "5326f3f1e0eb002bb14ce421d526f05c"
server_key_iv: "af2a3c935f8f46a55adcccec989af062"
server_key_version: "encryptionkey@0"
I20260812 06:56:54.159387 9419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:54.173626 9419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:54.178036 9419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:54.178355 9419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:54.179337 9419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:54.180787 9419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:54.180893 9419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:54.181048 9419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:54.181136 9419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:54.213766 9419 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:34555
I20260812 06:56:54.213812 9659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:34555 every 8 connection(s)
I20260812 06:56:54.215052 9419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:54.221661 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 9419
I20260812 06:56:54.221848 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:54.222352 27528 external_mini_cluster.cc:1428] Setting key 790cd9dbcac12a019b66ce0bff0cda76
Aug 12 06:56:54 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517814, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:56:54.235383 9664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:54.217640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:55357 (local address 127.26.226.62:34153)
0812 06:56:54.218122 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:54.218127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:54.218922 (+ 795us) server_negotiation.cc:407] Connection header received
0812 06:56:54.220851 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:54.220857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:54.220964 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:54.221089 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:54.222726 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:54.223849 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:54.225605 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:54.226007 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:54.226154 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:54.229625 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:54.229631 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:54.229634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:54.229668 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:54.232226 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:54.232970 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:54.232978 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:54.232987 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:54.233117 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:54.233595 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:54.233601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:54.233609 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:54.233866 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:54.234045 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:54.234722 (+ 677us) server_negotiation.cc:299] Negotiation successful
0812 06:56:54.234941 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":245,"threads_started":1}
I20260812 06:56:54.236440 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:54.218061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:55357)
0812 06:56:54.218759 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:54.218803 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:54.220581 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:54.221321 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:54.221329 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:54.221906 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:54.222552 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:54.222572 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:54.224009 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:54.224020 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:54.225436 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:54.225454 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:54.226247 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:54.226254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:54.226497 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:54.227132 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:54.227154 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:54.229396 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:54.232370 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:54.232379 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:54.232396 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:54.232823 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:54.233235 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:54.233240 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:54.233244 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:54.233436 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:54.234002 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:54.234010 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:54.234345 (+ 335us) client_negotiation.cc:769] Sending connection context
0812 06:56:54.234624 (+ 279us) client_negotiation.cc:240] Negotiation successful
0812 06:56:54.235156 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":269,"threads_started":1}
I20260812 06:56:54.237879 9660 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34153
I20260812 06:56:54.238224 9660 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:54.238925 9660 heartbeater.cc:511] Master 127.26.226.62:34153 requested a full tablet report, sending...
I20260812 06:56:54.240444 8827 ts_manager.cc:194] Registered new tserver with Master: bc5e239f0ea644fb8cc8ba5a2a81c16b (127.26.226.3:34555)
I20260812 06:56:54.241441 8827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:55357
I20260812 06:56:54.250288 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab.
Entry for principal oryx/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab.
WARNING: no policy specified for oryx/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab.
Entry for principal oryx/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab.
WARNING: no policy specified for oryx/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab.
Entry for principal oryx/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab.
I20260812 06:56:54.440984 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 8947
Aug 12 06:56:54 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:56:54.458120 9679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:54.442309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.26.226.1:34859)
0812 06:56:54.442938 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:54.442945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:54.442974 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:56:54.443068 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:54.443074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:54.443300 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:54.443448 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:54.444892 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:54.445920 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:54.447218 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:54.447798 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:54.447905 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:54.452245 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:54.452257 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:54.452273 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:54.452308 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:54.455455 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:54.456203 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:54.456211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:54.456214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:54.456338 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:54.456669 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:54.456672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:54.456674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:54.457018 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:54.457258 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:54.457488 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 06:56:54.457686 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":307,"threads_started":1}
I20260812 06:56:54.459403 9163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:54.542868 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 8947
I20260812 06:56:54.565574 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 9182
Aug 12 06:56:54 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:56:54.582940 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:54.566609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.26.226.2:33337)
0812 06:56:54.567158 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:54.567166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:54.567194 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:56:54.567256 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:54.567263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:54.567476 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:54.567611 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:54.568913 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:54.569948 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:54.571487 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:54.572304 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:54.572453 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:54.577012 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:54.577025 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:54.577040 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:54.577076 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:54.579889 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:54.580718 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:54.580725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:54.580728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:54.580849 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:54.581256 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:54.581259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:54.581261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:54.581621 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:54.581882 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:54.582332 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 06:56:54.582590 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":271,"threads_started":1}
I20260812 06:56:54.584398 9401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:54.667539 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 9182
I20260812 06:56:54.690111 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 9419
Aug 12 06:56:54 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:56:54.708629 9683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:54.691343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.26.226.3:34555)
0812 06:56:54.691912 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:54.691920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:54.691952 (+ 32us) server_negotiation.cc:407] Connection header received
0812 06:56:54.692017 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:54.692047 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:54.692348 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:54.692485 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:54.694473 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:54.695511 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:54.697052 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:54.697667 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:54.697801 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:54.702754 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:54.702768 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:54.702786 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:54.702827 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:54.705791 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:54.706480 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:54.706488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:54.706490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:54.706627 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:54.706976 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:54.706980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:54.706982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:54.707355 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:54.707601 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:54.707920 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 06:56:54.708240 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":270,"threads_started":1}
I20260812 06:56:54.710067 9648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:54.801980 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 9419
I20260812 06:56:54.828238 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 8773
I20260812 06:56:54.829602 8884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:54.934041 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 8773
I20260812 06:56:54.964267 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34153
--webserver_interface=127.26.226.62
--webserver_port=45517
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:34153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:55.304894 9687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:55.305238 9687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:55.305392 9687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:55.312759 9687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:55.312914 9687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:55.313037 9687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:55.313143 9687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:55.313220 9687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:55.323359 9687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:34153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=45517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:55.324872 9687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:55.326231 9687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:55.338445 9703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:55.338577 9698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:55.338577 9699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:55.340169 9687 server_base.cc:1034] running on GCE node
I20260812 06:56:55.341043 9687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:55.342474 9687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:55.343691 9687 hybrid_clock.cc:648] HybridClock initialized: now 1786517815343638 us; error 44 us; skew 500 ppm
I20260812 06:56:55.344139 9687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:55 dist-test-slave-t3q3 krb5kdc[8753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517815, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:55.347470 9687 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:56:55.349138 9687 webserver.cc:460] Webserver started at http://127.26.226.62:45517/ using document root <none> and password file <none>
I20260812 06:56:55.349869 9687 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:55.350037 9687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:55.355790 9687 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 06:56:55.358752 9724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:55.360220 9687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 06:56:55.360422 9687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "c365ab48199f456a9e8b4cd411fafe8f"
format_stamp: "Formatted at 2026-08-12 06:56:52 on dist-test-slave-t3q3"
server_key: "df73fa87cec11b1e71dc8dbca9faf8d8"
server_key_iv: "9b8705de5ef3aa9a17b452c7b04401bd"
server_key_version: "encryptionkey@0"
I20260812 06:56:55.361460 9687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:55.373287 9687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:55.377794 9687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:55.378088 9687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:55.398506 9687 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:34153
I20260812 06:56:55.398556 9811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:34153 every 8 connection(s)
I20260812 06:56:55.400218 9687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:55.403291 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 9687
I20260812 06:56:55.404422 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:34859
--local_ip_for_outbound_sockets=127.26.226.1
--tserver_master_addrs=127.26.226.62:34153
--webserver_port=46517
--webserver_interface=127.26.226.1
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.226.1
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:56:55.409404 9816 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:56:55.414685 9816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f: Bootstrap starting.
I20260812 06:56:55.435024 9816 log.cc:824] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:55.444759 9816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:56:55.445345 9816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f: Bootstrap complete.
I20260812 06:56:55.454766 9816 raft_consensus.cc:348] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } }
I20260812 06:56:55.455673 9816 raft_consensus.cc:729] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c365ab48199f456a9e8b4cd411fafe8f, State: Initialized, Role: FOLLOWER
I20260812 06:56:55.456350 9816 consensus_queue.cc:260] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } }
I20260812 06:56:55.456622 9816 raft_consensus.cc:388] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:55.456763 9816 raft_consensus.cc:482] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:55.456954 9816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:56:55.461018 9816 raft_consensus.cc:504] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } }
I20260812 06:56:55.461529 9816 leader_election.cc:302] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c365ab48199f456a9e8b4cd411fafe8f; no voters:
I20260812 06:56:55.462230 9816 leader_election.cc:288] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:56:55.462383 9825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:56:55.463821 9825 raft_consensus.cc:686] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [term 2 LEADER]: Becoming Leader. State: Replica: c365ab48199f456a9e8b4cd411fafe8f, State: Running, Role: LEADER
I20260812 06:56:55.464464 9825 consensus_queue.cc:237] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } }
I20260812 06:56:55.464725 9816 sys_catalog.cc:564] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:55.467299 9827 sys_catalog.cc:455] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c365ab48199f456a9e8b4cd411fafe8f. Latest consensus state: current_term: 2 leader_uuid: "c365ab48199f456a9e8b4cd411fafe8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } } }
I20260812 06:56:55.467784 9827 sys_catalog.cc:458] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:55.468333 9826 sys_catalog.cc:455] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c365ab48199f456a9e8b4cd411fafe8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c365ab48199f456a9e8b4cd411fafe8f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34153 } } }
I20260812 06:56:55.468703 9826 sys_catalog.cc:458] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:55.476745 9837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:55.483135 9837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:55.484645 9837 catalog_manager.cc:1259] Loaded cluster ID: 303a71f91de344729e73594f3485d70d
I20260812 06:56:55.484835 9837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:55.492905 9837 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:55.494457 9837 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c365ab48199f456a9e8b4cd411fafe8f: Loaded TSK: 0
I20260812 06:56:55.496042 9837 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:56:55.790544 9820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:55.790910 9820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:55.791117 9820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:55.798725 9820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:55.798913 9820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:55.799137 9820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:56:55.809532 9820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.1.keytab
--principal=oryx/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:34859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=46517
--tserver_master_addrs=127.26.226.62:34153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:55.810973 9820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:55.812443 9820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:55.828213 9865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:55.828228 9867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:55.828440 9871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:55.829977 9820 server_base.cc:1034] running on GCE node
I20260812 06:56:55.830763 9820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:55.832101 9820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:55.833369 9820 hybrid_clock.cc:648] HybridClock initialized: now 1786517815833321 us; error 30 us; skew 500 ppm
I20260812 06:56:55.833757 9820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:55 dist-test-slave-t3q3 krb5kdc[8753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517815, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:55.838093 9820 init.cc:377] Logged in from keytab as oryx/127.26.226.1@KRBTEST.COM (short username oryx)
I20260812 06:56:55.840317 9820 webserver.cc:460] Webserver started at http://127.26.226.1:46517/ using document root <none> and password file <none>
I20260812 06:56:55.841305 9820 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:55.841476 9820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:55.849645 9820 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 06:56:55.853567 9884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:55.855293 9820 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:56:55.855476 9820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "ea7e6681f8094a0688f09d864d9b0349"
format_stamp: "Formatted at 2026-08-12 06:56:52 on dist-test-slave-t3q3"
server_key: "a51f6c2eab7398557dbbf5c2bf68587f"
server_key_iv: "ffdef2110876c0130995535746a1a69f"
server_key_version: "encryptionkey@0"
I20260812 06:56:55.856901 9820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:55.895016 9820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:55.899243 9820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:55.899487 9820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:55.900571 9820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:55.901878 9820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:55.901979 9820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:55.902067 9820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:55.902125 9820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:55.936864 9820 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:34859
I20260812 06:56:55.936901 10097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:34859 every 8 connection(s)
I20260812 06:56:55.938369 9820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:55.942480 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 9820
I20260812 06:56:55.943312 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:33337
--local_ip_for_outbound_sockets=127.26.226.2
--tserver_master_addrs=127.26.226.62:34153
--webserver_port=37235
--webserver_interface=127.26.226.2
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.226.2
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:56:55 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:55 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 06:56:56.303604 10109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:56.303890 10109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:56.304050 10109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:56.311677 10109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:56.311796 10109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:56.312021 10109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:56:56.322013 10109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.2.keytab
--principal=oryx/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:33337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=37235
--tserver_master_addrs=127.26.226.62:34153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:56.323472 10109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:56.324846 10109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:56.342523 10122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:56.342531 10121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:56.344024 10126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:56.344399 10109 server_base.cc:1034] running on GCE node
I20260812 06:56:56.345085 10109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:56.346118 10109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:56.347388 10109 hybrid_clock.cc:648] HybridClock initialized: now 1786517816347309 us; error 73 us; skew 500 ppm
I20260812 06:56:56.347771 10109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:56 dist-test-slave-t3q3 krb5kdc[8753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517816, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:56.351392 10109 init.cc:377] Logged in from keytab as oryx/127.26.226.2@KRBTEST.COM (short username oryx)
I20260812 06:56:56.353154 10109 webserver.cc:460] Webserver started at http://127.26.226.2:37235/ using document root <none> and password file <none>
I20260812 06:56:56.353945 10109 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:56.354120 10109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:56.360272 10109 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:56:56.363674 10136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:56.365630 10109 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:56:56.365844 10109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "6f014368642b4fdf89e428bc56c338a6"
format_stamp: "Formatted at 2026-08-12 06:56:53 on dist-test-slave-t3q3"
server_key: "f2dfc8e9a3e5af4157c9a152f2094362"
server_key_iv: "46cceebc608fcc60b14b3a69dc7892d5"
server_key_version: "encryptionkey@0"
I20260812 06:56:56.366940 10109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:56.380662 10109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:56.385131 10109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:56.385419 10109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:56.386530 10109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:56.387982 10109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:56.388140 10109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 06:56:56.388279 10109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:56.388371 10109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:56.439321 10109 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33337
I20260812 06:56:56.439375 10354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33337 every 8 connection(s)
I20260812 06:56:56.441111 10109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:56.445721 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 10109
I20260812 06:56:56.446777 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:34555
--local_ip_for_outbound_sockets=127.26.226.3
--tserver_master_addrs=127.26.226.62:34153
--webserver_port=40783
--webserver_interface=127.26.226.3
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.226.3
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:56:56 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:56 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 06:56:56.818789 10371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:56.819171 10371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:56.819358 10371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:56.828636 10371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:56.828799 10371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:56.828998 10371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:56:56.839231 10371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/krb5kdc/oryx_127.26.226.3.keytab
--principal=oryx/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:34555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=40783
--tserver_master_addrs=127.26.226.62:34153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:56:56.840875 10371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:56.842278 10371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:56.860870 10386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:56.868005 10387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:56.868646 10392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:56.869386 10371 server_base.cc:1034] running on GCE node
I20260812 06:56:56.870164 10371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:56.871513 10371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:56.872900 10371 hybrid_clock.cc:648] HybridClock initialized: now 1786517816872834 us; error 63 us; skew 500 ppm
I20260812 06:56:56.873256 10371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:56 dist-test-slave-t3q3 krb5kdc[8753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517816, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:56.876571 10371 init.cc:377] Logged in from keytab as oryx/127.26.226.3@KRBTEST.COM (short username oryx)
I20260812 06:56:56.878480 10371 webserver.cc:460] Webserver started at http://127.26.226.3:40783/ using document root <none> and password file <none>
I20260812 06:56:56.879220 10371 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:56.879388 10371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:56.885283 10371 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:56:56.888099 10409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:56.889570 10371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:56:56.889757 10371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "bc5e239f0ea644fb8cc8ba5a2a81c16b"
format_stamp: "Formatted at 2026-08-12 06:56:54 on dist-test-slave-t3q3"
server_key: "5326f3f1e0eb002bb14ce421d526f05c"
server_key_iv: "af2a3c935f8f46a55adcccec989af062"
server_key_version: "encryptionkey@0"
I20260812 06:56:56.890745 10371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:56.914565 10371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:56.918931 10371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:56.919217 10371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:56.920351 10371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:56.921802 10371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:56.921921 10371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:56.922063 10371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:56.922163 10371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:56.976828 10371 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:34555
I20260812 06:56:56.976955 10613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:34555 every 8 connection(s)
I20260812 06:56:56.979822 10371 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:56.986459 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 10371
Aug 12 06:56:57 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:57 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:57.072611 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:56.989131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48570 (local address 127.26.226.62:34153)
0812 06:56:57.032377 (+ 43246us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:57.032386 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:57.032414 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:56:57.032514 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:57.032519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:57.032645 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:57.032814 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:57.039306 (+ 6492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.041219 (+ 1913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.050286 (+ 9067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.051312 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.051467 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.057242 (+ 5775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:57.057261 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:57.057292 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:57.057363 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:57.063300 (+ 5937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:57.064860 (+ 1560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:57.064876 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:57.064891 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:57.065284 (+ 393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:57.066991 (+ 1707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:57.067001 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:57.067007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:57.067926 (+ 919us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:57.068441 (+ 515us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:57.070190 (+ 1749us) server_negotiation.cc:299] Negotiation successful
0812 06:56:57.070981 (+ 791us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43105,"thread_start_us":44337,"threads_started":1}
W20260812 06:56:57.387583 10106 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.432s user 0.006s sys 0.010s
W20260812 06:56:57.389253 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:55.941549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:52445 (local address 127.26.226.62:34153)
0812 06:56:55.942383 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:55.942401 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:55.943019 (+ 618us) server_negotiation.cc:407] Connection header received
0812 06:56:55.946280 (+ 3261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:55.946292 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:55.946639 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:55.946941 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:55.949078 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:55.950367 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:55.952945 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:55.953608 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:55.953753 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.388907 (+1435154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:52445: BlockingRecv error: recv got EOF from 127.26.226.1:52445 (error 108)
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":271,"threads_started":1}
I20260812 06:56:57.390183 10106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:55.941665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:52445)
0812 06:56:55.942801 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:55.942856 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:55.945718 (+ 2862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:55.947169 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:55.947181 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:55.947997 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:55.948841 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:55.948870 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:55.950544 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:55.950552 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:55.952781 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:55.952815 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:55.953902 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:55.953912 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:55.954244 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:55.955011 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:55.955046 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:57.388622 (+1433576us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":598,"threads_started":1}
W20260812 06:56:57.391224 10098 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:34153 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.226.62:34153: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Aug 12 06:56:57 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:57 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:57.410599 10106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:57.391985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:37027)
0812 06:56:57.394049 (+ 2064us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:57.394070 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:57.394465 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:57.395162 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:57.395165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:57.395533 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:57.395698 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:57.395707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.396889 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.396894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:57.397945 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:57.397960 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.399171 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.399177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:57.399350 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.404521 (+ 5171us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:57.404547 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:57.410258 (+ 5711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1911,"mutex_wait_us":1445}
W20260812 06:56:57.410732 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:57.392156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:37027 (local address 127.26.226.62:34153)
0812 06:56:57.392548 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:57.392553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:57.394135 (+ 1582us) server_negotiation.cc:407] Connection header received
0812 06:56:57.394638 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:57.394643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:57.394775 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:57.394854 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:57.395869 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.396732 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.398111 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.398727 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.398838 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.410480 (+ 11642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:37027: BlockingRecv error: recv got EOF from 127.26.226.1:37027 (error 108)
Metrics: {"server-negotiator.queue_time_us":277}
Aug 12 06:56:57 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:57 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:57.420710 10106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:57.411725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:41277)
0812 06:56:57.411880 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:57.411899 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:57.412024 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:57.412475 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:57.412482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:57.412847 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:57.413029 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:57.413041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.414083 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.414089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:57.415063 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:57.415075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.415807 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.415814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:57.416021 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.416670 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:57.416688 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:57.420430 (+ 3742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 06:56:57.420898 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:57.411968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:41277 (local address 127.26.226.62:34153)
0812 06:56:57.412179 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:57.412185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:57.412200 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:56:57.412254 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:57.412257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:57.412325 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:57.412433 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:57.413181 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.413932 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.415208 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.415640 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.415774 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.420630 (+ 4856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:41277: BlockingRecv error: recv got EOF from 127.26.226.1:41277 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 06:56:57.421283 10098 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:56:57.934191 10369 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.480s user 0.000s sys 0.016s
W20260812 06:56:57.935820 10365 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:56.444438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:40119 (local address 127.26.226.62:34153)
0812 06:56:56.445122 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:56.445132 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:56.445906 (+ 774us) server_negotiation.cc:407] Connection header received
0812 06:56:56.447868 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:56.447874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:56.447987 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:56.448137 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:56.449744 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:56.450530 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:56.452118 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:56.452564 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:56.452716 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.935609 (+1482893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:40119: BlockingRecv error: recv got EOF from 127.26.226.2:40119 (error 108)
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":320,"threads_started":1}
I20260812 06:56:57.937284 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:56.444711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:40119)
0812 06:56:56.445737 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:56.445784 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:56.447587 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:56.448489 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:56.448497 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:56.449013 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:56.449556 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:56.449572 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:56.450688 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:56.450698 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:56.451937 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:56.451949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:56.452801 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:56.452809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:56.453073 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:56.453790 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:56.453812 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:57.935513 (+1481701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":490,"threads_started":1}
W20260812 06:56:57.938344 10355 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:34153 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.226.62:34153: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Aug 12 06:56:57 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:57 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:57.951843 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:57.939174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:51119)
0812 06:56:57.939821 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:57.939850 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:57.940021 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:57.940462 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:57.940470 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:57.941171 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:57.941496 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:57.941513 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.942538 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.942547 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:57.944910 (+ 2363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:57.944932 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.945722 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.945732 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:57.946010 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.946820 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:57.946855 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:57.951623 (+ 4768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":185}
W20260812 06:56:57.952437 10365 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:57.939532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:51119 (local address 127.26.226.62:34153)
0812 06:56:57.939709 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:57.939713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:57.939936 (+ 223us) server_negotiation.cc:407] Connection header received
0812 06:56:57.940222 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:57.940226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:57.940287 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:57.940371 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:57.941661 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.942374 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.945107 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.945504 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.945603 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.952002 (+ 6399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:51119: BlockingRecv error: recv got EOF from 127.26.226.2:51119 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 06:56:57 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:57 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:57.963152 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:57.952614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:42193)
0812 06:56:57.952831 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:57.952850 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:57.952971 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:57.953608 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:57.953612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:57.953998 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:57.954171 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:57.954180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.956212 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.956217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:57.957317 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:57.957330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.958670 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.958674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:57.958802 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.959415 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:57.959433 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:57.962956 (+ 3523us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102}
W20260812 06:56:57.963482 10355 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:56:57.963778 10365 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:57.952857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:42193 (local address 127.26.226.62:34153)
0812 06:56:57.953147 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:57.953156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:57.953183 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:56:57.953276 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:57.953284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:57.953390 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:57.953537 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:57.954340 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.955971 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.957477 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.958500 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.958682 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.963368 (+ 4686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:42193: BlockingRecv error: recv got EOF from 127.26.226.2:42193 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
W20260812 06:56:58.337859 10618 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.274s user 0.009s sys 0.019s
W20260812 06:56:58.339286 10620 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:56.988614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:43209 (local address 127.26.226.62:34153)
0812 06:56:57.031895 (+ 43281us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:57.031906 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:57.036164 (+ 4258us) server_negotiation.cc:407] Connection header received
0812 06:56:57.051214 (+ 15050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:57.051225 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:57.051368 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:57.051576 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:57.055077 (+ 3501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.056863 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.060361 (+ 3498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.061466 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.061617 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:58.338888 (+1277271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:43209: BlockingRecv error: recv got EOF from 127.26.226.3:43209 (error 108)
Metrics: {"server-negotiator.queue_time_us":43080,"thread_start_us":44918,"threads_started":1}
I20260812 06:56:58.339684 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:56.985257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:43209)
0812 06:56:57.034629 (+ 49372us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:57.034713 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:57.048562 (+ 13849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:57.052431 (+ 3869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:57.052450 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:57.053575 (+ 1125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:57.054753 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:57.054788 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.057161 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.057170 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:57.060146 (+ 2976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:57.060165 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:57.062199 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:57.062210 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:57.062790 (+ 580us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:57.063804 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:57.063850 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:58.338692 (+1274842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48775,"thread_start_us":55982,"threads_started":1}
W20260812 06:56:58.340687 10615 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:34153 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.226.62:34153: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Aug 12 06:56:58 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:58 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:58.350332 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:58.341339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:43881)
0812 06:56:58.341537 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:58.341560 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:58.341676 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:58.341973 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:58.341976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:58.342352 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:58.342516 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:58.342525 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.343519 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.343523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:58.344561 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:58.344574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.345387 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.345391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:58.345513 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:58.346162 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:58.346179 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:58.350117 (+ 3938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 06:56:58.350515 10620 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:58.341369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:43881 (local address 127.26.226.62:34153)
0812 06:56:58.341548 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:58.341552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:58.341634 (+ 82us) server_negotiation.cc:407] Connection header received
0812 06:56:58.341778 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:58.341781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:58.341842 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:58.341929 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:58.342641 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.343387 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.344724 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.345216 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.345320 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:58.350307 (+ 4987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:43881: BlockingRecv error: recv got EOF from 127.26.226.3:43881 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 06:56:58 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:58 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:58.359707 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:58.351045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:56761)
0812 06:56:58.351208 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:58.351229 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:58.351337 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:58.351779 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:58.351786 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:58.352202 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:58.352355 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:58.352363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.353418 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.353424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:58.354373 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:58.354385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.355139 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.355146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:58.355312 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:58.355929 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:58.355945 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:58.359425 (+ 3480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 06:56:58.360045 10620 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:58.351221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:56761 (local address 127.26.226.62:34153)
0812 06:56:58.351490 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:58.351495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:58.351524 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:56:58.351563 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:58.351567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:58.351635 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:58.351742 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:58.352495 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.353270 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.354524 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.354968 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.355102 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:58.359627 (+ 4525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:56761: BlockingRecv error: recv got EOF from 127.26.226.3:56761 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
W20260812 06:56:58.360136 10615 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 06:56:58 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:58 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:58.431382 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:58.421925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:37549)
0812 06:56:58.422553 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:58.422571 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:58.422691 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:58.423023 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:58.423028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:58.423470 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:58.423643 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:58.423652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.424729 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.424733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:58.425707 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:58.425720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.426397 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.426401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:58.426537 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:58.427139 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:58.427156 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:58.431127 (+ 3971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":383,"threads_started":1}
W20260812 06:56:58.431480 10620 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:58.422061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:37549 (local address 127.26.226.62:34153)
0812 06:56:58.422259 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:58.422264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:58.422651 (+ 387us) server_negotiation.cc:407] Connection header received
0812 06:56:58.422794 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:58.422798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:58.422883 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:58.422972 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:58.423804 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.424571 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.425849 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.426233 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.426332 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:58.431316 (+ 4984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:37549: BlockingRecv error: recv got EOF from 127.26.226.1:37549 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 06:56:58 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:58 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:58.974479 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:58.964217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:44983)
0812 06:56:58.964805 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:58.964854 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:58.965034 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:58.965470 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:58.965474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:58.966128 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:58.966339 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:58.966352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.967658 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.967662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:58.968730 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:58.968742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.969506 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.969511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:58.969658 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:58.970272 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:58.970290 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:58.974238 (+ 3948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":338,"threads_started":1}
W20260812 06:56:58.974732 10627 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:58.964320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:44983 (local address 127.26.226.62:34153)
0812 06:56:58.964882 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:58.964887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:58.964996 (+ 109us) server_negotiation.cc:407] Connection header received
0812 06:56:58.965207 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:58.965213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:58.965288 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:58.965405 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:58.966533 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.967513 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.968858 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:58.969311 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:58.969412 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:58.974463 (+ 5051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:44983: BlockingRecv error: recv got EOF from 127.26.226.2:44983 (error 108)
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":318,"threads_started":1}
Aug 12 06:56:59 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:59 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:59.371371 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.360694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:43463)
0812 06:56:59.361275 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:59.361292 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:59.361418 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:59.361823 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:59.361827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:59.362366 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:59.362601 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:59.362612 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.363662 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.363667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:59.364810 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:59.364825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.365765 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.365771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:59.365928 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:59.366691 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:59.366710 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:59.371021 (+ 4311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":345,"threads_started":1}
W20260812 06:56:59.371824 10627 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:59.360890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:43463 (local address 127.26.226.62:34153)
0812 06:56:59.361152 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:59.361158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:59.361401 (+ 243us) server_negotiation.cc:407] Connection header received
0812 06:56:59.361582 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:59.361587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:59.361655 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:59.361742 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:59.362778 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.363518 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.364953 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.365476 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.365623 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:59.371269 (+ 5646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:43463: BlockingRecv error: recv got EOF from 127.26.226.3:43463 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 12 06:56:59 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:59 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:59.442713 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.432641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:52937)
0812 06:56:59.433195 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:59.433212 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:59.433330 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:59.433639 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:59.433642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:59.434057 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:59.434256 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:59.434266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.435291 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.435295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:59.436323 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:59.436338 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.437062 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.437067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:59.437219 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:59.437897 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:59.437917 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:59.442370 (+ 4453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":317,"threads_started":1}
W20260812 06:56:59.442916 10627 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:59.432712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:52937 (local address 127.26.226.62:34153)
0812 06:56:59.432888 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:59.432893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:59.433291 (+ 398us) server_negotiation.cc:407] Connection header received
0812 06:56:59.433416 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:59.433420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:59.433484 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:59.433573 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:59.434393 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.435159 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.436454 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.436859 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.436961 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:59.442629 (+ 5668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:52937: BlockingRecv error: recv got EOF from 127.26.226.1:52937 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 06:56:59 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:56:59 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:56:59.985920 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.975534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:55485)
0812 06:56:59.976259 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:59.976286 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:59.976441 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:59.976842 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:59.976846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:59.977341 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:59.977592 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:59.977603 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.979119 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.979123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:59.980170 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:59.980184 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.980879 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.980883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:59.981015 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:59.981626 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:59.981643 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:59.985671 (+ 4028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":348,"threads_started":1}
W20260812 06:56:59.986263 10631 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:56:59.975623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:55485 (local address 127.26.226.62:34153)
0812 06:56:59.976439 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:59.976445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:59.976467 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:56:59.976566 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:59.976571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:59.976664 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:59.976775 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:59.977754 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.978950 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.980313 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.980737 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.980836 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:59.985867 (+ 5031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:55485: BlockingRecv error: recv got EOF from 127.26.226.2:55485 (error 108)
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":534,"threads_started":1}
Aug 12 06:57:00 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:00 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:00.383104 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.372506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:37655)
0812 06:57:00.373291 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.373316 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.373482 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.373902 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.373908 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.374527 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.374752 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.374763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.375870 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.375877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.377329 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.377343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.378136 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.378142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.378323 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.378973 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.379018 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.382835 (+ 3817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":514,"threads_started":1}
W20260812 06:57:00.383548 10631 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:00.372712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:37655 (local address 127.26.226.62:34153)
0812 06:57:00.372943 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.372954 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.373407 (+ 453us) server_negotiation.cc:407] Connection header received
0812 06:57:00.373673 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.373678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.373747 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.373865 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.374920 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.375720 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.377475 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.377910 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.378072 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.383088 (+ 5016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:37655: BlockingRecv error: recv got EOF from 127.26.226.3:37655 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 06:57:00 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:00 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:00.454027 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.443736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:44675)
0812 06:57:00.444378 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.444396 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.444522 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.444852 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.444856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.445274 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.445460 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.445470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.446691 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.446696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.447983 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.447998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.449117 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.449121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.449259 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.449879 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.449897 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.453714 (+ 3817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":377,"threads_started":1}
W20260812 06:57:00.454105 10631 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:00.443855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:44675 (local address 127.26.226.62:34153)
0812 06:57:00.444113 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.444121 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.444476 (+ 355us) server_negotiation.cc:407] Connection header received
0812 06:57:00.444627 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.444630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.444712 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.444799 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.445613 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.446521 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.448190 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.448976 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.449084 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.453916 (+ 4832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:44675: BlockingRecv error: recv got EOF from 127.26.226.1:44675 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 06:57:00 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:00 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:00.996977 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.986982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:38089)
0812 06:57:00.987532 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.987549 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.987658 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.988120 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.988124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.988498 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.988676 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.988684 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.989721 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.989725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.990925 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.990937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.991715 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.991721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.991920 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.992674 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.992692 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.996625 (+ 3933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":282,"threads_started":1}
W20260812 06:57:00.997236 10643 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:00.987057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:38089 (local address 127.26.226.62:34153)
0812 06:57:00.987587 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.987592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.987609 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:57:00.987823 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.987828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.987903 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.988015 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.988834 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.989587 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.991109 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.991558 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.991700 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.996943 (+ 5243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:38089: BlockingRecv error: recv got EOF from 127.26.226.2:38089 (error 108)
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":292,"threads_started":1}
Aug 12 06:57:01 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:01 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:01.394484 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.384340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:60063)
0812 06:57:01.385565 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:01.385591 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:01.385748 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:01.386272 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:01.386278 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:01.386683 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:01.386860 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:01.386870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.387954 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.387959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:01.388900 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:01.388912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.389778 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.389782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:01.389920 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.390520 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:01.390538 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:01.394162 (+ 3624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1074,"thread_start_us":564,"threads_started":1}
W20260812 06:57:01.394527 10643 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.384512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:60063 (local address 127.26.226.62:34153)
0812 06:57:01.385699 (+ 1187us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.385933 (+ 234us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.385954 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:57:01.386011 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.386016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.386102 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.386228 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.387026 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.387791 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.389119 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.389614 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.389744 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.394357 (+ 4613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:60063: BlockingRecv error: recv got EOF from 127.26.226.3:60063 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 06:57:01 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:01 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:01.465723 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.455083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:54861)
0812 06:57:01.455711 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:01.455729 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:01.455908 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:01.456426 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:01.456434 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:01.456923 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:01.457105 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:01.457115 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.458307 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.458313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:01.459828 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:01.459843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.460782 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.460787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:01.460945 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.461638 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:01.461677 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:01.465327 (+ 3650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":359,"threads_started":1}
W20260812 06:57:01.465894 10643 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.455202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:54861 (local address 127.26.226.62:34153)
0812 06:57:01.455530 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.455555 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.455829 (+ 274us) server_negotiation.cc:407] Connection header received
0812 06:57:01.456053 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.456102 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.456220 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.456364 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.457263 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.458126 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.460002 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.460645 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.460747 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.465632 (+ 4885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:54861: BlockingRecv error: recv got EOF from 127.26.226.1:54861 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Aug 12 06:57:02 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:02 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:02.008172 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.998291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:58941)
0812 06:57:01.998857 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:01.998875 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:01.998986 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:01.999408 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:01.999411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:01.999796 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:01.999972 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:01.999981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.001248 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.001253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:02.002276 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:02.002288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.003185 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.003190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:02.003392 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.004167 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:02.004187 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:02.007725 (+ 3538us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":302,"threads_started":1}
W20260812 06:57:02.008272 10651 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.998380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:58941 (local address 127.26.226.62:34153)
0812 06:57:01.998910 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.998915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.998934 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:57:01.999154 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.999159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.999242 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.999346 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:02.000188 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.001106 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.002422 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.002966 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.003176 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.007973 (+ 4797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:58941: BlockingRecv error: recv got EOF from 127.26.226.2:58941 (error 108)
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":285,"threads_started":1}
Aug 12 06:57:02 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:02 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:02.407310 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:02.395632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:43067)
0812 06:57:02.396399 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:02.396418 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:02.396619 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:02.397040 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:02.397045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:02.397512 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:02.397694 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:02.397704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.399144 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.399150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:02.400805 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:02.400823 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.401569 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.401576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:02.401767 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.402478 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:02.402500 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:02.406940 (+ 4440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":475,"threads_started":1}
W20260812 06:57:02.407382 10651 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:02.395868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:43067 (local address 127.26.226.62:34153)
0812 06:57:02.396179 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:02.396190 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:02.396535 (+ 345us) server_negotiation.cc:407] Connection header received
0812 06:57:02.396787 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:02.396792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:02.396869 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:02.396996 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:02.397877 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.398934 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.400945 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.401415 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.401531 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.407177 (+ 5646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:43067: BlockingRecv error: recv got EOF from 127.26.226.3:43067 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 06:57:02 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:02 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:02.480479 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:02.466788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:42937)
0812 06:57:02.467334 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:02.467351 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:02.467481 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:02.467835 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:02.467838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:02.468376 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:02.468534 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:02.468544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.469561 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.469565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:02.470569 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:02.470590 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.471285 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.471289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:02.471422 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.476579 (+ 5157us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:02.476600 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:02.480225 (+ 3625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":315,"threads_started":1}
W20260812 06:57:02.480729 10651 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:02.466913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:42937 (local address 127.26.226.62:34153)
0812 06:57:02.467098 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:02.467103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:02.467430 (+ 327us) server_negotiation.cc:407] Connection header received
0812 06:57:02.467611 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:02.467615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:02.467693 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:02.467784 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:02.468669 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.469437 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.470726 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.471155 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.471252 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.480423 (+ 9171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:42937: BlockingRecv error: recv got EOF from 127.26.226.1:42937 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 06:57:03 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:03 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:03.019977 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:03.009209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:34995)
0812 06:57:03.009888 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:03.009913 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:03.010154 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:03.011165 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:03.011169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:03.011654 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:03.011812 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:03.011820 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.013755 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.013761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:03.014747 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:03.014759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.015467 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.015471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:03.015611 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:03.016314 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:03.016331 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:03.019750 (+ 3419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":360,"threads_started":1}
W20260812 06:57:03.020324 10657 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:03.009640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:34995 (local address 127.26.226.62:34153)
0812 06:57:03.010742 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:03.010747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:03.010774 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:57:03.010854 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:03.010857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:03.010974 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:03.011111 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:03.012105 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.013553 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.014887 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.015322 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.015442 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:03.019961 (+ 4519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:34995: BlockingRecv error: recv got EOF from 127.26.226.2:34995 (error 108)
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":778,"threads_started":1}
Aug 12 06:57:03 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:03 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:03.417553 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:03.408386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:53259)
0812 06:57:03.408955 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:03.408978 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:03.409133 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:03.409534 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:03.409539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:03.410148 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:03.410297 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:03.410305 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.411320 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.411324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:03.412365 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:03.412378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.413114 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.413118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:03.413236 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:03.413883 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:03.413900 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:03.417326 (+ 3426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":282,"threads_started":1}
W20260812 06:57:03.417784 10657 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:03.408459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:53259 (local address 127.26.226.62:34153)
0812 06:57:03.408684 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:03.408692 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:03.409073 (+ 381us) server_negotiation.cc:407] Connection header received
0812 06:57:03.409285 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:03.409290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:03.409388 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:03.409469 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:03.410429 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.411211 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.412520 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.412931 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.413074 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:03.417527 (+ 4453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:53259: BlockingRecv error: recv got EOF from 127.26.226.3:53259 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 06:57:03 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:03 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:03.491173 10660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:03.481657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:39507)
0812 06:57:03.482233 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:03.482250 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:03.482427 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:03.482918 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:03.482922 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:03.483337 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:03.483494 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:03.483503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.484535 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.484540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:03.485463 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:03.485475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.486300 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.486305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:03.486453 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:03.487113 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:03.487131 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:03.490795 (+ 3664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":304,"threads_started":1}
W20260812 06:57:03.491258 10657 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:03.481734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:39507 (local address 127.26.226.62:34153)
0812 06:57:03.481975 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:03.481985 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:03.482355 (+ 370us) server_negotiation.cc:407] Connection header received
0812 06:57:03.482625 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:03.482629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:03.482737 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:03.482841 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:03.483634 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.484404 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.485619 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:03.486065 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:03.486167 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:03.491003 (+ 4836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:39507: BlockingRecv error: recv got EOF from 127.26.226.1:39507 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 06:57:04 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:04 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:04.030416 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.021030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:50247)
0812 06:57:04.021579 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.021598 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:04.021717 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:04.022045 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:04.022048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:04.022471 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:04.022633 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.022643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.023621 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.023625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.024604 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.024617 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.025354 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.025358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.025488 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.026129 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:04.026147 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:04.030005 (+ 3858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":295,"threads_started":1}
W20260812 06:57:04.030447 10662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:04.021067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:50247 (local address 127.26.226.62:34153)
0812 06:57:04.021606 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:04.021611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:04.021669 (+ 58us) server_negotiation.cc:407] Connection header received
0812 06:57:04.021823 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:04.021826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:04.021903 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:04.021996 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:04.022767 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.023478 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.024751 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.025203 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.025300 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.030232 (+ 4932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:50247: BlockingRecv error: recv got EOF from 127.26.226.2:50247 (error 108)
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":287,"threads_started":1}
Aug 12 06:57:04 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:04 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:04.428460 10663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.418627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:54059)
0812 06:57:04.419223 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.419241 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:04.419408 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:04.419790 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:04.419796 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:04.420332 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:04.420503 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.420513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.421571 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.421578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.422520 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.422532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.423264 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.423270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.423460 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.424222 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:04.424241 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:04.428124 (+ 3883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":365,"threads_started":1}
W20260812 06:57:04.428646 10662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:04.418720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:54059 (local address 127.26.226.62:34153)
0812 06:57:04.419007 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:04.419016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:04.419339 (+ 323us) server_negotiation.cc:407] Connection header received
0812 06:57:04.419559 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:04.419566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:04.419637 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:04.419749 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:04.420664 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.421405 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.422673 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.423097 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.423240 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.428443 (+ 5203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:54059: BlockingRecv error: recv got EOF from 127.26.226.3:54059 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 12 06:57:04 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:04 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:04.502553 10666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.492277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:45579)
0812 06:57:04.492902 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.492919 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:04.493093 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:04.493481 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:04.493486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:04.493962 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:04.494156 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.494164 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.495360 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.495367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.496513 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.496526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.497262 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.497268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.497430 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.498041 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:04.498058 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:04.502193 (+ 4135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":273,"threads_started":1}
W20260812 06:57:04.502656 10662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:04.492349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:45579 (local address 127.26.226.62:34153)
0812 06:57:04.492551 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:04.492559 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:04.493016 (+ 457us) server_negotiation.cc:407] Connection header received
0812 06:57:04.493224 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:04.493229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:04.493328 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:04.493446 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:04.494336 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.495217 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.496673 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.497094 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.497233 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.502384 (+ 5151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:45579: BlockingRecv error: recv got EOF from 127.26.226.1:45579 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 06:57:05 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:05 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:05.041761 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.031664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:34141)
0812 06:57:05.032477 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.032494 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:05.032667 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:05.033121 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:05.033126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:05.033611 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:05.033801 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.033809 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.034870 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.034877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.035841 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.035852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.036662 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.036668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.036844 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.037649 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:05.037672 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:05.041510 (+ 3838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":450,"threads_started":1}
W20260812 06:57:05.042001 10670 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.031753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:34141 (local address 127.26.226.62:34153)
0812 06:57:05.032371 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.032379 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.032595 (+ 216us) server_negotiation.cc:407] Connection header received
0812 06:57:05.032830 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.032834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.032912 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.033049 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.033960 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.034723 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.035993 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.036496 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.036647 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.041734 (+ 5087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:34141: BlockingRecv error: recv got EOF from 127.26.226.2:34141 (error 108)
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":330,"threads_started":1}
Aug 12 06:57:05 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:05 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:05.443013 10674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.429616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:59131)
0812 06:57:05.430294 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.430312 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:05.430427 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:05.430748 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:05.430751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:05.431168 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:05.431342 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.431350 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.432374 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.432378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.433367 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.433380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.434020 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.434024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.434161 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.438738 (+ 4577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:05.438757 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:05.442668 (+ 3911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":321,"threads_started":1}
W20260812 06:57:05.443027 10670 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.429706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:59131 (local address 127.26.226.62:34153)
0812 06:57:05.429903 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.429907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.430390 (+ 483us) server_negotiation.cc:407] Connection header received
0812 06:57:05.430530 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.430534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.430595 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.430685 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.431473 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.432243 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.433495 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.433892 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.433982 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.442841 (+ 8859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:59131: BlockingRecv error: recv got EOF from 127.26.226.3:59131 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 12 06:57:05 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:05 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:05.513151 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.503597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:55155)
0812 06:57:05.504189 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.504211 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:05.504386 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:05.504761 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:05.504766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:05.505245 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:05.505426 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.505436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.506484 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.506491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.507675 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.507693 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.508636 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.508644 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.508802 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.509396 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:05.509413 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:05.512891 (+ 3478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":263,"threads_started":1}
W20260812 06:57:05.513381 10670 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.503857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:55155 (local address 127.26.226.62:34153)
0812 06:57:05.504159 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.504168 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.504313 (+ 145us) server_negotiation.cc:407] Connection header received
0812 06:57:05.504517 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.504522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.504616 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.504728 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.505583 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.506339 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.507830 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.508403 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.508548 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.513133 (+ 4585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:55155: BlockingRecv error: recv got EOF from 127.26.226.1:55155 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 12 06:57:06 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:06 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:06.052595 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:06.042910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:36497)
0812 06:57:06.043477 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:06.043495 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:06.043609 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:06.043972 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:06.043975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:06.044439 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:06.044614 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:06.044623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.046209 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.046215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:06.047350 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:06.047364 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.048110 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.048118 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:06.048259 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:06.048843 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:06.048861 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:06.052347 (+ 3486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":284,"threads_started":1}
W20260812 06:57:06.052778 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:06.043019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:36497 (local address 127.26.226.62:34153)
0812 06:57:06.043558 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:06.043563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:06.043585 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:57:06.043745 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:06.043749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:06.043824 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:06.043923 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:06.044782 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.046034 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.047490 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.047898 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.047993 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:06.052542 (+ 4549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:36497: BlockingRecv error: recv got EOF from 127.26.226.2:36497 (error 108)
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":329,"threads_started":1}
Aug 12 06:57:06 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:06 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:06.453228 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:06.444277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:44585)
0812 06:57:06.444852 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:06.444869 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:06.445029 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:06.445482 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:06.445491 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:06.446005 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:06.446177 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:06.446187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.447215 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.447219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:06.448170 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:06.448182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.448854 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.448858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:06.448965 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:06.449540 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:06.449556 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:06.452946 (+ 3390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":312,"threads_started":1}
W20260812 06:57:06.453315 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:06.444369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:44585 (local address 127.26.226.62:34153)
0812 06:57:06.444573 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:06.444581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:06.444962 (+ 381us) server_negotiation.cc:407] Connection header received
0812 06:57:06.445181 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:06.445187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:06.445289 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:06.445403 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:06.446320 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.447089 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.448297 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.448694 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.448791 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:06.453140 (+ 4349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:44585: BlockingRecv error: recv got EOF from 127.26.226.3:44585 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 06:57:06 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:06 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:06.523924 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:06.514280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:55209)
0812 06:57:06.514890 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:06.514907 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:06.515024 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:06.515422 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:06.515427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:06.516022 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:06.516276 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:06.516291 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.517541 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.517545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:06.518506 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:06.518518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.519186 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.519191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:06.519323 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:06.519900 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:06.519917 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:06.523574 (+ 3657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":329,"threads_started":1}
W20260812 06:57:06.524032 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:06.514339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:55209 (local address 127.26.226.62:34153)
0812 06:57:06.514521 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:06.514525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:06.514989 (+ 464us) server_negotiation.cc:407] Connection header received
0812 06:57:06.515161 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:06.515165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:06.515241 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:06.515326 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:06.516431 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.517403 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.518638 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:06.519036 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:06.519143 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:06.523779 (+ 4636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:55209: BlockingRecv error: recv got EOF from 127.26.226.1:55209 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 06:57:07 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:07 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:07.063462 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:07.053708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:60569)
0812 06:57:07.054208 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:07.054226 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:07.054335 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:07.054785 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:07.054789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:07.055238 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:07.055391 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:07.055400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.056387 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.056391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:07.057328 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:07.057340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.058124 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.058132 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:07.058331 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.059027 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:07.059081 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:07.063117 (+ 4036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":247,"threads_started":1}
W20260812 06:57:07.063604 10688 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:07.053763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:60569 (local address 127.26.226.62:34153)
0812 06:57:07.054498 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:07.054502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:07.054518 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:57:07.054586 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:07.054589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:07.054648 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:07.054738 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:07.055521 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.056269 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.057475 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.057947 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.058091 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.063311 (+ 5220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:60569: BlockingRecv error: recv got EOF from 127.26.226.2:60569 (error 108)
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":513,"threads_started":1}
Aug 12 06:57:07 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:07 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:07.464351 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:07.454274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:46197)
0812 06:57:07.454814 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:07.454830 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:07.454943 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:07.455276 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:07.455280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:07.455770 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:07.455925 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:07.455934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.457411 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.457416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:07.458687 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:07.458701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.459374 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.459379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:07.459509 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.460158 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:07.460176 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:07.463953 (+ 3777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":290,"threads_started":1}
W20260812 06:57:07.464494 10688 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:07.454329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:46197 (local address 127.26.226.62:34153)
0812 06:57:07.454513 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:07.454518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:07.454911 (+ 393us) server_negotiation.cc:407] Connection header received
0812 06:57:07.455067 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:07.455071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:07.455134 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:07.455227 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:07.456156 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.457226 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.458836 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.459233 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.459331 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.464217 (+ 4886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:46197: BlockingRecv error: recv got EOF from 127.26.226.3:46197 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 06:57:07 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:07 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:07.538064 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:07.524975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:56877)
0812 06:57:07.525561 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:07.525580 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:07.525711 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:07.526057 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:07.526060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:07.526477 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:07.526668 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:07.526677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.527675 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.527679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:07.528938 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:07.528955 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.530054 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.530062 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:07.530235 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.534096 (+ 3861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:07.534114 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:07.537805 (+ 3691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":299,"threads_started":1}
W20260812 06:57:07.538350 10688 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:07.525191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:56877 (local address 127.26.226.62:34153)
0812 06:57:07.525461 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:07.525472 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:07.525692 (+ 220us) server_negotiation.cc:407] Connection header received
0812 06:57:07.525821 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:07.525825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:07.525904 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:07.525993 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:07.526808 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.527535 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.529133 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.529814 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.529962 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.538032 (+ 8070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:56877: BlockingRecv error: recv got EOF from 127.26.226.1:56877 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
Aug 12 06:57:08 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:08 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:08.074129 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:08.064511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:40287)
0812 06:57:08.065096 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:08.065118 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:08.065255 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:08.065675 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:08.065679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:08.066193 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:08.066355 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:08.066364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.067421 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.067425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:08.068428 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:08.068440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.069156 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.069162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:08.069293 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:08.069894 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:08.069913 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:08.073884 (+ 3971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":291,"threads_started":1}
W20260812 06:57:08.074455 10693 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:08.064624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:40287 (local address 127.26.226.62:34153)
0812 06:57:08.065229 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:08.065235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:08.065255 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:57:08.065414 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:08.065419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:08.065511 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:08.065636 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:08.066491 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.067291 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.068560 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.068976 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.069074 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:08.074159 (+ 5085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:40287: BlockingRecv error: recv got EOF from 127.26.226.2:40287 (error 108)
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":320,"threads_started":1}
Aug 12 06:57:08 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:08 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:08.474375 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:08.465492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:47027)
0812 06:57:08.466093 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:08.466115 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:08.466243 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:08.466538 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:08.466541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:08.466907 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:08.467075 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:08.467083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.468158 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.468162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:08.469150 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:08.469163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.469806 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.469812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:08.469926 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:08.470579 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:08.470597 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:08.474133 (+ 3536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":340,"threads_started":1}
W20260812 06:57:08.474519 10693 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:08.465644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:47027 (local address 127.26.226.62:34153)
0812 06:57:08.465922 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:08.465930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:08.466211 (+ 281us) server_negotiation.cc:407] Connection header received
0812 06:57:08.466351 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:08.466354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:08.466419 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:08.466494 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:08.467220 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.467969 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.469277 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.469672 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.469770 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:08.474319 (+ 4549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:47027: BlockingRecv error: recv got EOF from 127.26.226.3:47027 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
Aug 12 06:57:08 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:08 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:08.547665 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:08.539051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:57135)
0812 06:57:08.539509 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:08.539527 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:08.539647 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:08.540005 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:08.540009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:08.540507 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:08.540712 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:08.540723 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.541732 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.541736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:08.542654 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:08.542665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.543303 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.543307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:08.543427 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:08.544033 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:08.544050 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:08.547432 (+ 3382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":254,"threads_started":1}
W20260812 06:57:08.547863 10693 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:08.539101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:57135 (local address 127.26.226.62:34153)
0812 06:57:08.539371 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:08.539377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:08.539611 (+ 234us) server_negotiation.cc:407] Connection header received
0812 06:57:08.539752 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:08.539756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:08.539832 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:08.539918 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:08.540855 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.541607 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.542779 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:08.543164 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:08.543263 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:08.547616 (+ 4353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:57135: BlockingRecv error: recv got EOF from 127.26.226.1:57135 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 06:57:09 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:09 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:09.084982 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:09.075103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:47769)
0812 06:57:09.075612 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:09.075630 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:09.075738 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:09.076281 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:09.076285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:09.076724 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:09.076909 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:09.076918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.078060 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.078064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:09.079060 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:09.079073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.079921 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.079927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:09.080175 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:09.080939 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:09.080957 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:09.084648 (+ 3691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":265,"threads_started":1}
W20260812 06:57:09.085122 10697 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:09.075220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:47769 (local address 127.26.226.62:34153)
0812 06:57:09.075913 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:09.075917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:09.075933 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:57:09.075985 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:09.075989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:09.076116 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:09.076239 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:09.077061 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.077939 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.079192 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.079733 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.079861 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:09.084858 (+ 4997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:47769: BlockingRecv error: recv got EOF from 127.26.226.2:47769 (error 108)
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":484,"threads_started":1}
Aug 12 06:57:09 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:09 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:09.485379 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:09.475315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:39399)
0812 06:57:09.475888 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:09.475904 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:09.476114 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:09.476527 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:09.476532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:09.477025 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:09.477240 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:09.477250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.478451 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.478458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:09.479471 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:09.479484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.480493 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.480501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:09.480719 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:09.481441 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:09.481460 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:09.485149 (+ 3689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":306,"threads_started":1}
W20260812 06:57:09.485561 10697 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:09.475567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:39399 (local address 127.26.226.62:34153)
0812 06:57:09.475802 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:09.475809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:09.475991 (+ 182us) server_negotiation.cc:407] Connection header received
0812 06:57:09.476298 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:09.476310 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:09.476377 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:09.476491 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:09.477396 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.478307 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.479624 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.480273 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.480405 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:09.485342 (+ 4937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:39399: BlockingRecv error: recv got EOF from 127.26.226.3:39399 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 12 06:57:09 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:09 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:09.558293 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:09.548695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:54465)
0812 06:57:09.549213 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:09.549230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:09.549360 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:09.549778 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:09.549782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:09.550193 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:09.550373 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:09.550382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.551454 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.551458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:09.552503 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:09.552516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.553301 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.553305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:09.553462 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:09.554154 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:09.554172 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:09.558033 (+ 3861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":300,"threads_started":1}
W20260812 06:57:09.558480 10697 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:09.548789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:54465 (local address 127.26.226.62:34153)
0812 06:57:09.549067 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:09.549074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:09.549322 (+ 248us) server_negotiation.cc:407] Connection header received
0812 06:57:09.549532 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:09.549537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:09.549620 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:09.549717 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:09.550542 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.551317 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.552663 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:09.553167 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:09.553274 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:09.558215 (+ 4941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:54465: BlockingRecv error: recv got EOF from 127.26.226.1:54465 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 12 06:57:10 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:10 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:10.094831 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:10.086082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:34389)
0812 06:57:10.086633 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:10.086650 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:10.086775 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:10.087095 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:10.087098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:10.087516 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:10.087694 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:10.087720 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.088789 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.088793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:10.089736 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:10.089748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.090414 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.090418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:10.090554 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:10.091180 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:10.091197 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:10.094569 (+ 3372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":304,"threads_started":1}
W20260812 06:57:10.095022 10706 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:10.086122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:34389 (local address 127.26.226.62:34153)
0812 06:57:10.086638 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:10.086643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:10.086729 (+ 86us) server_negotiation.cc:407] Connection header received
0812 06:57:10.086882 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:10.086886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:10.086959 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:10.087051 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:10.087868 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.088665 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.089863 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.090287 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.090380 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:10.094783 (+ 4403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:34389: BlockingRecv error: recv got EOF from 127.26.226.2:34389 (error 108)
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":285,"threads_started":1}
Aug 12 06:57:10 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:10 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:10.496441 10707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:10.486485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:49133)
0812 06:57:10.487058 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:10.487079 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:10.487249 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:10.487627 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:10.487631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:10.488243 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:10.488440 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:10.488450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.489525 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.489531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:10.490497 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:10.490510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.491287 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.491293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:10.491496 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:10.492198 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:10.492225 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:10.496168 (+ 3943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":288,"threads_started":1}
W20260812 06:57:10.496570 10706 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:10.486563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:49133 (local address 127.26.226.62:34153)
0812 06:57:10.486788 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:10.486795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:10.487215 (+ 420us) server_negotiation.cc:407] Connection header received
0812 06:57:10.487395 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:10.487401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:10.487478 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:10.487590 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:10.488597 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.489370 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.490656 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.491107 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.491264 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:10.496365 (+ 5101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:49133: BlockingRecv error: recv got EOF from 127.26.226.3:49133 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 06:57:10 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:10 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:10.569262 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:10.559377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:34419)
0812 06:57:10.559955 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:10.559973 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:10.560168 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:10.560560 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:10.560564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:10.561134 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:10.561353 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:10.561367 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.562572 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.562578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:10.563642 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:10.563657 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.564382 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.564388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:10.564599 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:10.565323 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:10.565341 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:10.568884 (+ 3543us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":353,"threads_started":1}
W20260812 06:57:10.569612 10706 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:10.559503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:34419 (local address 127.26.226.62:34153)
0812 06:57:10.559772 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:10.559781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:10.560094 (+ 313us) server_negotiation.cc:407] Connection header received
0812 06:57:10.560334 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:10.560338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:10.560414 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:10.560523 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:10.561506 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.562418 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.563794 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:10.564221 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:10.564366 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:10.569089 (+ 4723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:34419: BlockingRecv error: recv got EOF from 127.26.226.1:34419 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 06:57:11 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:11 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.2@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:11.105118 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:11.095820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.2:51823)
0812 06:57:11.096448 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:11.096466 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:11.096593 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:11.096953 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:11.096957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:11.097387 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:11.097536 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:11.097574 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.098883 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.098889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:11.099896 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:11.099908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.100671 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.100677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:11.100874 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:11.101477 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:11.101494 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:11.104854 (+ 3360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":344,"threads_started":1}
W20260812 06:57:11.105324 10712 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:11.096003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:51823 (local address 127.26.226.62:34153)
0812 06:57:11.096513 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:11.096518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:11.096535 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:57:11.096710 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:11.096713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:11.096783 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:11.096912 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:11.097725 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.098728 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.100091 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.100508 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.100649 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:11.105052 (+ 4403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.2:51823: BlockingRecv error: recv got EOF from 127.26.226.2:51823 (error 108)
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":281,"threads_started":1}
Aug 12 06:57:11 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:11 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.3@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:11.506794 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:11.497467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.3:33149)
0812 06:57:11.497956 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:11.497978 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:11.498098 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:11.498546 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:11.498551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:11.499173 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:11.499351 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:11.499361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.500565 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.500571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:11.501478 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:11.501489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.502168 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.502172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:11.502295 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:11.502887 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:11.502904 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:11.506480 (+ 3576us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":272,"threads_started":1}
W20260812 06:57:11.506906 10712 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:11.497535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:33149 (local address 127.26.226.62:34153)
0812 06:57:11.497802 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:11.497809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:11.498066 (+ 257us) server_negotiation.cc:407] Connection header received
0812 06:57:11.498236 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:11.498241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:11.498322 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:11.498443 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:11.499521 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.500429 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.501608 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.502015 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.502110 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:11.506671 (+ 4561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.3:33149: BlockingRecv error: recv got EOF from 127.26.226.3:33149 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 12 06:57:11 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:57:11 dist-test-slave-t3q3 krb5kdc[8753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.226.1@KRBTEST.COM for oryx/127.26.226.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:57:11.580327 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:11.570589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34153 (local address 127.26.226.1:46119)
0812 06:57:11.571239 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:11.571287 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:11.571440 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:11.571876 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:11.571881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:11.572384 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:11.572568 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:11.572580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.573633 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.573640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:11.574682 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:11.574695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.575401 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.575406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:11.575593 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:11.576239 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:11.576257 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:11.579992 (+ 3735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.226.62:34153: Server oryx/127.26.226.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":324,"threads_started":1}
W20260812 06:57:11.580477 10712 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:11.570797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:46119 (local address 127.26.226.62:34153)
0812 06:57:11.571153 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:11.571161 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:11.571369 (+ 208us) server_negotiation.cc:407] Connection header received
0812 06:57:11.571630 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:11.571636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:11.571736 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:11.571839 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:11.572721 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.573479 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.574822 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.575242 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.575375 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:11.580241 (+ 4866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.226.1:46119: BlockingRecv error: recv got EOF from 127.26.226.1:46119 (error 108)
Metrics: {"server-negotiator.queue_time_us":186}
I20260812 06:57:11.990614 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 9820
I20260812 06:57:12.003562 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:11.991898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.26.226.1:34859)
0812 06:57:11.992750 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:11.992758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:11.992790 (+ 32us) server_negotiation.cc:407] Connection header received
0812 06:57:11.992857 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:11.992863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:11.993137 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:11.993309 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:11.994794 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.995984 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.997204 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:11.997755 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:11.997859 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:12.000408 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:12.000417 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:12.000432 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:12.000472 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:12.003007 (+ 2535us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.226.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 06:57:12.003313 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60710: Request ticket server kudu/127.26.226.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":500,"threads_started":1}
W20260812 06:57:12.003700 10717 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60710: Request ticket server kudu/127.26.226.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 06:57:12.003712 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 9820: Not authorized: Client connection negotiation failed: client connection to 127.26.226.1:34859: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.226.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 06:57:12.003949 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 9820
I20260812 06:57:12.034775 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 10109
I20260812 06:57:12.047816 10720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:12.036029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.26.226.2:33337)
0812 06:57:12.036714 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:12.036721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:12.036749 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:57:12.036818 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:12.036825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:12.037074 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:12.037335 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:12.038743 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:12.039775 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:12.041146 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:12.041727 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:12.041883 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:12.044536 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:12.044545 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:12.044558 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:12.044599 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:12.047050 (+ 2451us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.226.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 06:57:12.047427 (+ 377us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53902: Request ticket server kudu/127.26.226.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":335,"threads_started":1}
W20260812 06:57:12.047868 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 10109: Not authorized: Client connection negotiation failed: client connection to 127.26.226.2:33337: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.226.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 06:57:12.047945 10720 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53902: Request ticket server kudu/127.26.226.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 06:57:12.048018 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 10109
I20260812 06:57:12.077816 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 10371
I20260812 06:57:12.091361 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:12.079148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.26.226.3:34555)
0812 06:57:12.079723 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:12.079731 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:12.079758 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:57:12.079829 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:12.079836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:12.080148 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:12.080343 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:12.082386 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:12.083437 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:12.084845 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:12.085397 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:12.085502 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:12.087988 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:12.087997 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:12.088013 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:12.088156 (+ 143us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:12.090750 (+ 2594us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.226.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 06:57:12.091031 (+ 281us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58632: Request ticket server kudu/127.26.226.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":258,"threads_started":1}
W20260812 06:57:12.091482 10722 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58632: Request ticket server kudu/127.26.226.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 06:57:12.091459 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 10371: Not authorized: Client connection negotiation failed: client connection to 127.26.226.3:34555: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.226.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 06:57:12.091665 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 10371
I20260812 06:57:12.120138 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 9687
I20260812 06:57:12.121655 9804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:12.239718 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 9687
2026-08-12T06:57:12Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21981 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T06:57:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:57:12Z Disabled control of system clock
I20260812 06:57:12.336369 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:45589
--webserver_interface=127.26.226.62
--webserver_port=0
--builtin_ntp_servers=127.26.226.20:34885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:45589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 06:57:12.681756 10733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:12.682111 10733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:12.682277 10733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:12.689738 10733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:12.689865 10733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:12.689904 10733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:12.690028 10733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:12.690106 10733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:12.700320 10733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:45589
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:45589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:12.701804 10733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:12.703552 10733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:12.715669 10741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:12.715749 10738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:12.715770 10746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:12.716499 10733 server_base.cc:1034] running on GCE node
I20260812 06:57:12.717377 10733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:12.718849 10733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:12.720137 10733 hybrid_clock.cc:648] HybridClock initialized: now 1786517832720027 us; error 53 us; skew 500 ppm
I20260812 06:57:12.720497 10733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:57:12.725550 10733 webserver.cc:460] Webserver started at http://127.26.226.62:44001/ using document root <none> and password file <none>
I20260812 06:57:12.726284 10733 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:12.726423 10733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:12.726769 10733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:12.730659 10733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "1aa1023e491f4e52b08612910845e1de"
format_stamp: "Formatted at 2026-08-12 06:57:12 on dist-test-slave-t3q3"
server_key: "808ecc6d69ea7d0389f6aadb9e64ef96"
server_key_iv: "05c0ece954c07efa0f5184de149fb83b"
server_key_version: "encryptionkey@0"
I20260812 06:57:12.731756 10733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "1aa1023e491f4e52b08612910845e1de"
format_stamp: "Formatted at 2026-08-12 06:57:12 on dist-test-slave-t3q3"
server_key: "808ecc6d69ea7d0389f6aadb9e64ef96"
server_key_iv: "05c0ece954c07efa0f5184de149fb83b"
server_key_version: "encryptionkey@0"
I20260812 06:57:12.738818 10733 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260812 06:57:12.742219 10757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:12.743569 10733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:57:12.743718 10733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "1aa1023e491f4e52b08612910845e1de"
format_stamp: "Formatted at 2026-08-12 06:57:12 on dist-test-slave-t3q3"
server_key: "808ecc6d69ea7d0389f6aadb9e64ef96"
server_key_iv: "05c0ece954c07efa0f5184de149fb83b"
server_key_version: "encryptionkey@0"
I20260812 06:57:12.743906 10733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:12.760833 10733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:12.761646 10733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:12.761905 10733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:12.780614 10733 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:45589
I20260812 06:57:12.780637 10843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:45589 every 8 connection(s)
I20260812 06:57:12.782080 10733 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:12.783550 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 10733
I20260812 06:57:12.783767 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:57:12.784677 27528 external_mini_cluster.cc:1428] Setting key aaa4e64743c05729a3dc80f1b44ec5bc
I20260812 06:57:12.786880 10844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:12.798352 10851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:12.786880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.26.226.62:45589)
0812 06:57:12.787859 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:12.787869 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:12.787910 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:57:12.789566 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:12.789586 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:12.790049 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:12.790461 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:12.791134 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:12.792540 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:12.793900 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:12.794551 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:12.794752 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:12.796017 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:12.796028 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:12.796041 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:12.796676 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:12.796756 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:12.796777 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:12.796943 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:12.797168 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 06:57:12.797463 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":360,"threads_started":1}
I20260812 06:57:12.799099 10844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de: Bootstrap starting.
I20260812 06:57:12.802218 10844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:12.803205 10844 log.cc:824] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:12.806159 10844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de: No bootstrap required, opened a new log
I20260812 06:57:12.813766 10844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa1023e491f4e52b08612910845e1de" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45589 } }
I20260812 06:57:12.814157 10844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:12.814280 10844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa1023e491f4e52b08612910845e1de, State: Initialized, Role: FOLLOWER
I20260812 06:57:12.814816 10844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa1023e491f4e52b08612910845e1de" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45589 } }
I20260812 06:57:12.815066 10844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:57:12.815217 10844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:57:12.815403 10844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:12.817628 10844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa1023e491f4e52b08612910845e1de" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45589 } }
I20260812 06:57:12.818179 10844 leader_election.cc:302] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aa1023e491f4e52b08612910845e1de; no voters:
I20260812 06:57:12.818778 10844 leader_election.cc:288] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:57:12.818938 10853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:12.820267 10853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [term 1 LEADER]: Becoming Leader. State: Replica: 1aa1023e491f4e52b08612910845e1de, State: Running, Role: LEADER
I20260812 06:57:12.820811 10853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa1023e491f4e52b08612910845e1de" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45589 } }
I20260812 06:57:12.821053 10844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:12.823793 10856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aa1023e491f4e52b08612910845e1de. Latest consensus state: current_term: 1 leader_uuid: "1aa1023e491f4e52b08612910845e1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa1023e491f4e52b08612910845e1de" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45589 } } }
I20260812 06:57:12.824319 10856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:12.824195 10855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aa1023e491f4e52b08612910845e1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa1023e491f4e52b08612910845e1de" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 45589 } } }
I20260812 06:57:12.824523 10855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:12.825879 10868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:12.831107 10868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:12.840204 10868 catalog_manager.cc:1347] Generated new cluster ID: f9f8819f5a0248b99b41577139081df3
I20260812 06:57:12.840327 10868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:12.854980 10868 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:57:12.856456 10868 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:12.869709 10868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aa1023e491f4e52b08612910845e1de: Generated new TSK 0
I20260812 06:57:12.870697 10868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:57:12.888223 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:45589
--builtin_ntp_servers=127.26.226.20:34885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 06:57:13.250136 10882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:13.250424 10882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:13.250511 10882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:13.257906 10882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:13.258014 10882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:13.258119 10882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:57:13.267406 10882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:45589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:13.268693 10882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:13.270467 10882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:13.283789 10890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:13.283682 10895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:13.283731 10892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:13.285100 10882 server_base.cc:1034] running on GCE node
I20260812 06:57:13.285794 10882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:13.286724 10882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:13.287933 10882 hybrid_clock.cc:648] HybridClock initialized: now 1786517833287879 us; error 44 us; skew 500 ppm
I20260812 06:57:13.288300 10882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:57:13.290709 10882 webserver.cc:460] Webserver started at http://127.26.226.1:40305/ using document root <none> and password file <none>
I20260812 06:57:13.291453 10882 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:13.291577 10882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:13.291941 10882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:13.294845 10882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "7df9ddab12d049fabe7a4084648827d0"
format_stamp: "Formatted at 2026-08-12 06:57:13 on dist-test-slave-t3q3"
server_key: "086c12fc6d73b7a143a510dfd145e64a"
server_key_iv: "9f33d8bfb6f0ca6de48bf044d5eab4d2"
server_key_version: "encryptionkey@0"
I20260812 06:57:13.295662 10882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "7df9ddab12d049fabe7a4084648827d0"
format_stamp: "Formatted at 2026-08-12 06:57:13 on dist-test-slave-t3q3"
server_key: "086c12fc6d73b7a143a510dfd145e64a"
server_key_iv: "9f33d8bfb6f0ca6de48bf044d5eab4d2"
server_key_version: "encryptionkey@0"
I20260812 06:57:13.300897 10882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260812 06:57:13.304523 10901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:13.305864 10882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:57:13.306048 10882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "7df9ddab12d049fabe7a4084648827d0"
format_stamp: "Formatted at 2026-08-12 06:57:13 on dist-test-slave-t3q3"
server_key: "086c12fc6d73b7a143a510dfd145e64a"
server_key_iv: "9f33d8bfb6f0ca6de48bf044d5eab4d2"
server_key_version: "encryptionkey@0"
I20260812 06:57:13.306272 10882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:13.320686 10882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:13.321597 10882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:13.321894 10882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:13.322973 10882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:13.324576 10882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:13.324689 10882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:13.324818 10882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:13.324924 10882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:13.357816 10882 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:36705
I20260812 06:57:13.357854 11097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:36705 every 8 connection(s)
I20260812 06:57:13.359207 10882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:57:13.361759 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 10882
I20260812 06:57:13.361979 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:57:13.362831 27528 external_mini_cluster.cc:1428] Setting key 224638d647599d8b698f3af5fb6fcc60
I20260812 06:57:13.365974 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:45589
--builtin_ntp_servers=127.26.226.20:34885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 06:57:13.374336 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:13.362116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:60683 (local address 127.26.226.62:45589)
0812 06:57:13.362609 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:13.362614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:13.364195 (+ 1581us) server_negotiation.cc:407] Connection header received
0812 06:57:13.366375 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:13.366382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:13.366471 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:13.366594 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:13.367639 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:13.369025 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:13.370907 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:13.371631 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:13.371803 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:13.373309 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:13.373318 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:13.373325 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:13.373351 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:13.373406 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:13.373425 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:13.373534 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:13.373766 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 06:57:13.373881 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":265,"threads_started":1}
I20260812 06:57:13.375202 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:13.362810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:45589 (local address 127.26.226.1:60683)
0812 06:57:13.363976 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:13.364029 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:13.366022 (+ 1993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:13.366807 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:13.366815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:13.366900 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:13.367460 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:13.367476 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:13.369247 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:13.369256 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:13.370742 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:13.370755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:13.371895 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:13.371905 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:13.372220 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:13.372956 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:57:13.372959 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:13.373012 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:57:13.373098 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:13.373537 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:13.373545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:13.373551 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:57:13.373721 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 06:57:13.374014 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":329,"threads_started":1}
I20260812 06:57:13.376883 11099 heartbeater.cc:347] Connected to a master server at 127.26.226.62:45589
I20260812 06:57:13.377208 11099 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:13.378018 11099 heartbeater.cc:511] Master 127.26.226.62:45589 requested a full tablet report, sending...
I20260812 06:57:13.380373 10789 ts_manager.cc:194] Registered new tserver with Master: 7df9ddab12d049fabe7a4084648827d0 (127.26.226.1:36705)
I20260812 06:57:13.382581 10789 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.1:60683
W20260812 06:57:13.714095 11107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:13.714389 11107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:13.714476 11107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:13.721939 11107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:13.722039 11107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:13.722144 11107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:57:13.731688 11107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:45589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:13.733088 11107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:13.734835 11107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:13.748907 11117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:13.748970 11114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:13.749325 11121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:13.750869 11107 server_base.cc:1034] running on GCE node
I20260812 06:57:13.751431 11107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:13.752465 11107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:13.753749 11107 hybrid_clock.cc:648] HybridClock initialized: now 1786517833753694 us; error 43 us; skew 500 ppm
I20260812 06:57:13.754089 11107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:57:13.756840 11107 webserver.cc:460] Webserver started at http://127.26.226.2:38099/ using document root <none> and password file <none>
I20260812 06:57:13.757579 11107 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:13.757721 11107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:13.758072 11107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:13.760990 11107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "ecc383247a2d4615bba526f3f4386845"
format_stamp: "Formatted at 2026-08-12 06:57:13 on dist-test-slave-t3q3"
server_key: "c03d1c48f19a5d5dc48c7427092ba2cf"
server_key_iv: "87f8a1605502b60d2cc1cf8a07eb1a34"
server_key_version: "encryptionkey@0"
I20260812 06:57:13.761811 11107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "ecc383247a2d4615bba526f3f4386845"
format_stamp: "Formatted at 2026-08-12 06:57:13 on dist-test-slave-t3q3"
server_key: "c03d1c48f19a5d5dc48c7427092ba2cf"
server_key_iv: "87f8a1605502b60d2cc1cf8a07eb1a34"
server_key_version: "encryptionkey@0"
I20260812 06:57:13.766855 11107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 06:57:13.770376 11129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:13.771663 11107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:57:13.771859 11107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "ecc383247a2d4615bba526f3f4386845"
format_stamp: "Formatted at 2026-08-12 06:57:13 on dist-test-slave-t3q3"
server_key: "c03d1c48f19a5d5dc48c7427092ba2cf"
server_key_iv: "87f8a1605502b60d2cc1cf8a07eb1a34"
server_key_version: "encryptionkey@0"
I20260812 06:57:13.772121 11107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:13.804641 11107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:13.805557 11107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:13.805889 11107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:13.806910 11107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:13.808681 11107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:13.808818 11107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:13.808959 11107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:13.809053 11107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:13.845633 11107 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:37599
I20260812 06:57:13.845732 11336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:37599 every 8 connection(s)
I20260812 06:57:13.847147 11107 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:57:13.849174 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 11107
I20260812 06:57:13.849359 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:57:13.849992 27528 external_mini_cluster.cc:1428] Setting key ea173662dbb07777eea65e0d230188e5
I20260812 06:57:13.853133 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:45589
--builtin_ntp_servers=127.26.226.20:34885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 06:57:13.861291 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:13.850207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:38877 (local address 127.26.226.62:45589)
0812 06:57:13.850389 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:13.850394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:13.852133 (+ 1739us) server_negotiation.cc:407] Connection header received
0812 06:57:13.854165 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:13.854173 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:13.854281 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:13.854399 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:13.855456 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:13.856625 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:13.858150 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:13.858853 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:13.858999 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:13.860370 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:13.860378 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:13.860386 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:13.860412 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:13.860458 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:13.860477 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:13.860575 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:13.860780 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 06:57:13.860892 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:57:13.862211 11345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:13.850747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:45589 (local address 127.26.226.2:38877)
0812 06:57:13.851936 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:13.851991 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:13.853877 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:13.854591 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:13.854599 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:13.854688 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:13.855279 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:13.855296 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:13.856768 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:13.856772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:13.857975 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:13.857987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:13.859081 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:13.859086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:13.859276 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:13.859993 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:57:13.859995 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:13.860041 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:57:13.860177 (+ 136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:13.860579 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:13.860588 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:13.860593 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:57:13.860738 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 06:57:13.861118 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":361,"threads_started":1}
I20260812 06:57:13.863737 11337 heartbeater.cc:347] Connected to a master server at 127.26.226.62:45589
I20260812 06:57:13.864104 11337 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:13.864864 11337 heartbeater.cc:511] Master 127.26.226.62:45589 requested a full tablet report, sending...
I20260812 06:57:13.866317 10789 ts_manager.cc:194] Registered new tserver with Master: ecc383247a2d4615bba526f3f4386845 (127.26.226.2:37599)
I20260812 06:57:13.867144 10789 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.2:38877
W20260812 06:57:14.208572 11346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:14.208918 11346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:14.209087 11346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:14.216343 11346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:14.216481 11346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:14.216657 11346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:57:14.225823 11346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:45589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:14.227113 11346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:14.228945 11346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:14.242233 11358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:14.242302 11353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:14.242326 11355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:14.243367 11346 server_base.cc:1034] running on GCE node
I20260812 06:57:14.243980 11346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:14.244992 11346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:14.246240 11346 hybrid_clock.cc:648] HybridClock initialized: now 1786517834246195 us; error 34 us; skew 500 ppm
I20260812 06:57:14.246573 11346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:57:14.249368 11346 webserver.cc:460] Webserver started at http://127.26.226.3:41963/ using document root <none> and password file <none>
I20260812 06:57:14.250110 11346 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:14.250258 11346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:14.250607 11346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:14.253638 11346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "0b93d2782ac848eda089aa09ada6067c"
format_stamp: "Formatted at 2026-08-12 06:57:14 on dist-test-slave-t3q3"
server_key: "73a4e2a7d45edbc3f55d42d72590b3b3"
server_key_iv: "dd69eac574310b165a3b63d8ac6c2749"
server_key_version: "encryptionkey@0"
I20260812 06:57:14.254570 11346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "0b93d2782ac848eda089aa09ada6067c"
format_stamp: "Formatted at 2026-08-12 06:57:14 on dist-test-slave-t3q3"
server_key: "73a4e2a7d45edbc3f55d42d72590b3b3"
server_key_iv: "dd69eac574310b165a3b63d8ac6c2749"
server_key_version: "encryptionkey@0"
I20260812 06:57:14.260040 11346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 06:57:14.263468 11368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:14.264849 11346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 06:57:14.265033 11346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "0b93d2782ac848eda089aa09ada6067c"
format_stamp: "Formatted at 2026-08-12 06:57:14 on dist-test-slave-t3q3"
server_key: "73a4e2a7d45edbc3f55d42d72590b3b3"
server_key_iv: "dd69eac574310b165a3b63d8ac6c2749"
server_key_version: "encryptionkey@0"
I20260812 06:57:14.265269 11346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:14.294657 11346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:14.295579 11346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:14.295903 11346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:14.296986 11346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:14.298514 11346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:14.298641 11346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:14.298789 11346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:14.298889 11346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:14.334102 11346 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:36981
I20260812 06:57:14.334199 11549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:36981 every 8 connection(s)
I20260812 06:57:14.335474 11346 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:57:14.336146 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 11346
I20260812 06:57:14.336364 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:57:14.336977 27528 external_mini_cluster.cc:1428] Setting key 598ec88dfe74f1e9df7768fd0fba9999
I20260812 06:57:14.349076 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:14.339107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:57317 (local address 127.26.226.62:45589)
0812 06:57:14.339314 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:14.339320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:14.340368 (+ 1048us) server_negotiation.cc:407] Connection header received
0812 06:57:14.342272 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:14.342277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:14.342362 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:14.342489 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:14.343593 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:14.344431 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:14.346014 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:14.346414 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:14.346562 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:14.348105 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:14.348112 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:14.348117 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:14.348176 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:14.348216 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:14.348233 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:14.348341 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:14.348586 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 06:57:14.348687 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:57:14.350060 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:14.338999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:45589 (local address 127.26.226.3:57317)
0812 06:57:14.340206 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:14.340260 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:14.341997 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:14.342733 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:14.342740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:14.342838 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:14.343410 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:14.343428 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:14.344586 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:14.344591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:14.345839 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:14.345853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:14.346653 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:14.346661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:14.346908 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:14.347733 (+ 825us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:57:14.347735 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:14.347775 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:57:14.347854 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:14.348344 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:14.348359 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:14.348368 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:57:14.348575 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 06:57:14.349008 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":405,"threads_started":1}
I20260812 06:57:14.351696 11555 heartbeater.cc:347] Connected to a master server at 127.26.226.62:45589
I20260812 06:57:14.352042 11555 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:14.353084 11555 heartbeater.cc:511] Master 127.26.226.62:45589 requested a full tablet report, sending...
I20260812 06:57:14.354501 10789 ts_manager.cc:194] Registered new tserver with Master: 0b93d2782ac848eda089aa09ada6067c (127.26.226.3:36981)
I20260812 06:57:14.355321 10789 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.3:57317
I20260812 06:57:14.365643 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 06:57:14.372311 11104 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:14.370618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41132 (local address 127.26.226.62:45589)
0812 06:57:14.370789 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:14.370793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:14.371350 (+ 557us) server_negotiation.cc:407] Connection header received
0812 06:57:14.371493 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:14.371497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:14.371572 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:14.371658 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:14.372027 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41132: BlockingRecv error: recv got EOF from 127.0.0.1:41132 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:57:14.373199 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 10882
I20260812 06:57:14.381024 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:14.374181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.26.226.1:36705)
0812 06:57:14.374719 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:14.374726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:14.374752 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:57:14.374808 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:14.374814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:14.375017 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:14.375158 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:14.375504 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:14.376719 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:14.377905 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:14.378445 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:14.378542 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:14.379561 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:14.379580 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:14.379594 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:14.380227 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:14.380274 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:14.380294 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:14.380395 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:14.380519 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 06:57:14.380682 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":262,"threads_started":1}
I20260812 06:57:14.382297 11087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:14.465485 11099 heartbeater.cc:503] Master 127.26.226.62:45589 was elected leader, sending a full tablet report...
I20260812 06:57:14.466687 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 10882
W20260812 06:57:14.490943 10763 connection.cc:535] server connection from 127.26.226.1:60683 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:57:14.491261 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 11107
I20260812 06:57:14.499496 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:14.492211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39640 (local address 127.26.226.2:37599)
0812 06:57:14.492757 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:14.492764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:14.492792 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:57:14.492854 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:14.492860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:14.493092 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:14.493294 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:14.493730 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:14.494712 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:14.495990 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:14.496663 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:14.496825 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:14.497950 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:14.497959 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:14.497974 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:14.498606 (+ 632us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:14.498655 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:14.498676 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:14.498822 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:14.499008 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 06:57:14.499145 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":274,"threads_started":1}
I20260812 06:57:14.500873 11330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:14.584187 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 11107
I20260812 06:57:14.614439 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 11346
I20260812 06:57:14.623260 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:14.615459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37346 (local address 127.26.226.3:36981)
0812 06:57:14.616177 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:14.616184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:14.616223 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:57:14.616292 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:14.616298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:14.616513 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:14.616700 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:14.617210 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:14.618273 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:14.619578 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:14.620314 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:14.620466 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:14.621649 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:14.621658 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:14.621673 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:14.622357 (+ 684us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:14.622407 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:14.622442 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:14.622577 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:14.622747 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 06:57:14.622894 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":341,"threads_started":1}
I20260812 06:57:14.624637 11542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:14.708345 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 11346
I20260812 06:57:14.743026 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 10733
I20260812 06:57:14.744433 10837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:14.849973 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 10733
2026-08-12T06:57:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2604 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T06:57:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:57:14Z Disabled control of system clock
I20260812 06:57:14.943485 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33119
--webserver_interface=127.26.226.62
--webserver_port=0
--builtin_ntp_servers=127.26.226.20:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:33119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 06:57:15.289463 11589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:15.289815 11589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:15.289903 11589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:15.297079 11589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:15.297227 11589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:15.297266 11589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:15.297326 11589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:15.297370 11589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:15.306581 11589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:33119
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:15.307835 11589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:15.309648 11589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:15.321722 11609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:15.321790 11604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:15.321821 11600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:15.322765 11589 server_base.cc:1034] running on GCE node
I20260812 06:57:15.323611 11589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:15.325101 11589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:15.326359 11589 hybrid_clock.cc:648] HybridClock initialized: now 1786517835326328 us; error 57 us; skew 500 ppm
I20260812 06:57:15.326725 11589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:57:15.329092 11589 webserver.cc:460] Webserver started at http://127.26.226.62:34411/ using document root <none> and password file <none>
I20260812 06:57:15.329870 11589 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:15.330016 11589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:15.330389 11589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:15.335232 11589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "071297faf8884d3589253d9cc0588a7b"
format_stamp: "Formatted at 2026-08-12 06:57:15 on dist-test-slave-t3q3"
server_key: "d990d8d459288c0d673dc50ccd97fae5"
server_key_iv: "216be42277b267dbb81b7aca151f91fe"
server_key_version: "encryptionkey@0"
I20260812 06:57:15.336104 11589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "071297faf8884d3589253d9cc0588a7b"
format_stamp: "Formatted at 2026-08-12 06:57:15 on dist-test-slave-t3q3"
server_key: "d990d8d459288c0d673dc50ccd97fae5"
server_key_iv: "216be42277b267dbb81b7aca151f91fe"
server_key_version: "encryptionkey@0"
I20260812 06:57:15.341568 11589 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 06:57:15.345242 11617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:15.346720 11589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:57:15.346940 11589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "071297faf8884d3589253d9cc0588a7b"
format_stamp: "Formatted at 2026-08-12 06:57:15 on dist-test-slave-t3q3"
server_key: "d990d8d459288c0d673dc50ccd97fae5"
server_key_iv: "216be42277b267dbb81b7aca151f91fe"
server_key_version: "encryptionkey@0"
I20260812 06:57:15.347198 11589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:15.353328 11589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:15.354419 11589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:15.354758 11589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:15.375438 11589 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:33119
I20260812 06:57:15.375456 11695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:33119 every 8 connection(s)
I20260812 06:57:15.377074 11589 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:15.381078 11696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:15.381600 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 11589
I20260812 06:57:15.381839 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:57:15.382714 27528 external_mini_cluster.cc:1428] Setting key f3baf2fe7302a6274d17ef26e7bdd0cf
I20260812 06:57:15.392269 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:15.384964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.26.226.62:33119)
0812 06:57:15.386093 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:15.386104 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:15.386150 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:57:15.388262 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:15.388296 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:15.388904 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:15.389298 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:15.390511 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:15.390517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:15.390530 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:15.390545 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:15.390621 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:15.390637 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:15.390807 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:15.390944 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 06:57:15.391235 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":416,"threads_started":1}
I20260812 06:57:15.395768 11696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b: Bootstrap starting.
I20260812 06:57:15.398976 11696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:15.400027 11696 log.cc:824] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:15.402968 11696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b: No bootstrap required, opened a new log
I20260812 06:57:15.410418 11696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071297faf8884d3589253d9cc0588a7b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33119 } }
I20260812 06:57:15.410825 11696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:15.410933 11696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 071297faf8884d3589253d9cc0588a7b, State: Initialized, Role: FOLLOWER
I20260812 06:57:15.411542 11696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071297faf8884d3589253d9cc0588a7b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33119 } }
I20260812 06:57:15.411787 11696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:57:15.411933 11696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:57:15.412130 11696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:15.414352 11696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071297faf8884d3589253d9cc0588a7b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33119 } }
I20260812 06:57:15.414894 11696 leader_election.cc:302] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 071297faf8884d3589253d9cc0588a7b; no voters:
I20260812 06:57:15.415585 11696 leader_election.cc:288] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:57:15.415725 11704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:15.417320 11704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [term 1 LEADER]: Becoming Leader. State: Replica: 071297faf8884d3589253d9cc0588a7b, State: Running, Role: LEADER
I20260812 06:57:15.417954 11704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071297faf8884d3589253d9cc0588a7b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33119 } }
I20260812 06:57:15.418129 11696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:15.421206 11705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "071297faf8884d3589253d9cc0588a7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071297faf8884d3589253d9cc0588a7b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33119 } } }
I20260812 06:57:15.422024 11705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:15.421446 11706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 071297faf8884d3589253d9cc0588a7b. Latest consensus state: current_term: 1 leader_uuid: "071297faf8884d3589253d9cc0588a7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071297faf8884d3589253d9cc0588a7b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33119 } } }
I20260812 06:57:15.422580 11706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:15.425305 11717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:15.430552 11717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:15.439491 11717 catalog_manager.cc:1347] Generated new cluster ID: fc8e86ef98534c8abc89ab61d468b144
I20260812 06:57:15.439632 11717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:15.454667 11717 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:57:15.455155 11717 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:15.461666 11717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 071297faf8884d3589253d9cc0588a7b: Generated new TSK 0
I20260812 06:57:15.462502 11717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:57:15.470804 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33119
--builtin_ntp_servers=127.26.226.20:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 06:57:15.877911 11739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:15.878187 11739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:15.878279 11739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:15.885632 11739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:15.885742 11739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:15.885905 11739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:57:15.895679 11739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33119
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:15.897099 11739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:15.899471 11739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:15.913820 11747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:15.913833 11748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:15.913735 11750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:15.914812 11739 server_base.cc:1034] running on GCE node
I20260812 06:57:15.915436 11739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:15.916585 11739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:15.917892 11739 hybrid_clock.cc:648] HybridClock initialized: now 1786517835917805 us; error 96 us; skew 500 ppm
I20260812 06:57:15.918251 11739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:57:15.920691 11739 webserver.cc:460] Webserver started at http://127.26.226.1:35737/ using document root <none> and password file <none>
I20260812 06:57:15.921365 11739 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:15.921458 11739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:15.921736 11739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:15.924728 11739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "cdda6259ae3c4711ae789242f14f78cc"
format_stamp: "Formatted at 2026-08-12 06:57:15 on dist-test-slave-t3q3"
server_key: "e7f73a6ed262e2634349a661c43154fb"
server_key_iv: "42ff1a185b809e0ccfc4f0629e3d7a70"
server_key_version: "encryptionkey@0"
I20260812 06:57:15.925539 11739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "cdda6259ae3c4711ae789242f14f78cc"
format_stamp: "Formatted at 2026-08-12 06:57:15 on dist-test-slave-t3q3"
server_key: "e7f73a6ed262e2634349a661c43154fb"
server_key_iv: "42ff1a185b809e0ccfc4f0629e3d7a70"
server_key_version: "encryptionkey@0"
I20260812 06:57:15.930575 11739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:57:15.933883 11762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:15.935194 11739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:57:15.935371 11739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "cdda6259ae3c4711ae789242f14f78cc"
format_stamp: "Formatted at 2026-08-12 06:57:15 on dist-test-slave-t3q3"
server_key: "e7f73a6ed262e2634349a661c43154fb"
server_key_iv: "42ff1a185b809e0ccfc4f0629e3d7a70"
server_key_version: "encryptionkey@0"
I20260812 06:57:15.935604 11739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:15.941351 11739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:15.942121 11739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:15.942396 11739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:15.943284 11739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:15.944730 11739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:15.944855 11739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:15.944988 11739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:15.945082 11739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:15.979100 11739 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:34603
I20260812 06:57:15.979156 11968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:34603 every 8 connection(s)
I20260812 06:57:15.980521 11739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:57:15.987921 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 11739
I20260812 06:57:15.988191 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:57:15.988958 27528 external_mini_cluster.cc:1428] Setting key cddd1044f848c84969638c4bee1b7ed1
I20260812 06:57:15.989054 11975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:15.983169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:49983 (local address 127.26.226.62:33119)
0812 06:57:15.983680 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:15.983685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:15.984733 (+ 1048us) server_negotiation.cc:407] Connection header received
0812 06:57:15.986482 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:15.986487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:15.986573 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:15.986696 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:15.988120 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:15.988129 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:15.988134 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:15.988162 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:15.988206 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:15.988219 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:15.988353 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:15.988636 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 06:57:15.988707 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":280,"threads_started":1}
I20260812 06:57:15.990017 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:15.983616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33119 (local address 127.26.226.1:49983)
0812 06:57:15.984556 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:15.984604 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:15.986210 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:15.986942 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:15.986950 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:15.987038 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:15.987791 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:57:15.987794 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:15.987830 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:57:15.987915 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:15.988324 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:15.988335 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:15.988365 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:57:15.988707 (+ 342us) client_negotiation.cc:240] Negotiation successful
0812 06:57:15.988957 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":272,"threads_started":1}
I20260812 06:57:15.991520 11970 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33119
I20260812 06:57:15.991870 11970 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:15.992144 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33119
--builtin_ntp_servers=127.26.226.20:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 06:57:15.992667 11970 heartbeater.cc:511] Master 127.26.226.62:33119 requested a full tablet report, sending...
I20260812 06:57:15.994899 11644 ts_manager.cc:194] Registered new tserver with Master: cdda6259ae3c4711ae789242f14f78cc (127.26.226.1:34603)
W20260812 06:57:16.340942 11977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:16.341280 11977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:16.341456 11977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:16.348935 11977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:16.349088 11977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:16.349277 11977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:57:16.358917 11977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33119
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:16.360355 11977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:16.362202 11977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:16.376622 11985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:16.376585 11986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:16.377051 11989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:16.381662 11977 server_base.cc:1034] running on GCE node
I20260812 06:57:16.382409 11977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:16.383486 11977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:16.384732 11977 hybrid_clock.cc:648] HybridClock initialized: now 1786517836384668 us; error 60 us; skew 500 ppm
I20260812 06:57:16.385093 11977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:57:16.387645 11977 webserver.cc:460] Webserver started at http://127.26.226.2:36489/ using document root <none> and password file <none>
I20260812 06:57:16.388480 11977 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:16.388628 11977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:16.388983 11977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:16.392238 11977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "cd133a968af64680a5a3a6844cc1e237"
format_stamp: "Formatted at 2026-08-12 06:57:16 on dist-test-slave-t3q3"
server_key: "8025c1fe91080fdbedf8c8e5a93e77f2"
server_key_iv: "955e583c557ad554a974cd5decd07afc"
server_key_version: "encryptionkey@0"
I20260812 06:57:16.393087 11977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "cd133a968af64680a5a3a6844cc1e237"
format_stamp: "Formatted at 2026-08-12 06:57:16 on dist-test-slave-t3q3"
server_key: "8025c1fe91080fdbedf8c8e5a93e77f2"
server_key_iv: "955e583c557ad554a974cd5decd07afc"
server_key_version: "encryptionkey@0"
I20260812 06:57:16.399040 11977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:57:16.402592 11999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:16.404155 11977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:57:16.404348 11977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "cd133a968af64680a5a3a6844cc1e237"
format_stamp: "Formatted at 2026-08-12 06:57:16 on dist-test-slave-t3q3"
server_key: "8025c1fe91080fdbedf8c8e5a93e77f2"
server_key_iv: "955e583c557ad554a974cd5decd07afc"
server_key_version: "encryptionkey@0"
I20260812 06:57:16.404598 11977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:16.410581 11977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:16.411409 11977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:16.411692 11977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:16.412758 11977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:16.414224 11977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:16.414361 11977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:16.414487 11977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:16.414577 11977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:16.450436 11977 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:34283
I20260812 06:57:16.450543 12190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:34283 every 8 connection(s)
I20260812 06:57:16.451993 11977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:57:16.453550 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 11977
I20260812 06:57:16.453763 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:57:16.454203 27528 external_mini_cluster.cc:1428] Setting key aa0febd4bb2225f1c7d2e2cf83145dd8
I20260812 06:57:16.457557 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33119
--builtin_ntp_servers=127.26.226.20:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 06:57:16.462294 11975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:16.454867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:35691 (local address 127.26.226.62:33119)
0812 06:57:16.455113 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:16.455120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:16.456914 (+ 1794us) server_negotiation.cc:407] Connection header received
0812 06:57:16.459668 (+ 2754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:16.459673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:16.459726 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:16.459821 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:16.461419 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:16.461424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:16.461427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:16.461445 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:16.461479 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:16.461491 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:16.461620 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:16.461884 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 06:57:16.461955 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 06:57:16.463388 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:16.455542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33119 (local address 127.26.226.2:35691)
0812 06:57:16.456719 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:16.456769 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:16.459346 (+ 2577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:16.460131 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:16.460145 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:16.460273 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:16.461060 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:57:16.461066 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:16.461117 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:57:16.461234 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:16.461597 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:16.461610 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:16.461620 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 06:57:16.461917 (+ 297us) client_negotiation.cc:240] Negotiation successful
0812 06:57:16.462182 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":414,"threads_started":1}
I20260812 06:57:16.464833 12193 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33119
I20260812 06:57:16.465166 12193 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:16.465778 12193 heartbeater.cc:511] Master 127.26.226.62:33119 requested a full tablet report, sending...
I20260812 06:57:16.467242 11644 ts_manager.cc:194] Registered new tserver with Master: cd133a968af64680a5a3a6844cc1e237 (127.26.226.2:34283)
W20260812 06:57:16.812230 12197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:16.812517 12197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:16.812610 12197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:16.820262 12197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:16.820374 12197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:16.820484 12197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:57:16.829885 12197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33119
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:16.831184 12197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:16.833003 12197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:16.846060 12204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:16.846161 12209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:16.846248 12206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:16.847043 12197 server_base.cc:1034] running on GCE node
I20260812 06:57:16.847673 12197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:16.848742 12197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:16.849989 12197 hybrid_clock.cc:648] HybridClock initialized: now 1786517836849918 us; error 49 us; skew 500 ppm
I20260812 06:57:16.850319 12197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:57:16.852937 12197 webserver.cc:460] Webserver started at http://127.26.226.3:38389/ using document root <none> and password file <none>
I20260812 06:57:16.853683 12197 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:16.853822 12197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:16.854161 12197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:16.857074 12197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "5306c30147464628b47c005c6f166306"
format_stamp: "Formatted at 2026-08-12 06:57:16 on dist-test-slave-t3q3"
server_key: "0ea5561af382d55bffe815bac15f552a"
server_key_iv: "ccbd2ea470ef467e2d2adaee50488407"
server_key_version: "encryptionkey@0"
I20260812 06:57:16.857877 12197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "5306c30147464628b47c005c6f166306"
format_stamp: "Formatted at 2026-08-12 06:57:16 on dist-test-slave-t3q3"
server_key: "0ea5561af382d55bffe815bac15f552a"
server_key_iv: "ccbd2ea470ef467e2d2adaee50488407"
server_key_version: "encryptionkey@0"
I20260812 06:57:16.863822 12197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:57:16.867645 12217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:16.869093 12197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:16.869285 12197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "5306c30147464628b47c005c6f166306"
format_stamp: "Formatted at 2026-08-12 06:57:16 on dist-test-slave-t3q3"
server_key: "0ea5561af382d55bffe815bac15f552a"
server_key_iv: "ccbd2ea470ef467e2d2adaee50488407"
server_key_version: "encryptionkey@0"
I20260812 06:57:16.869531 12197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:16.875698 12197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:16.876601 12197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:16.876905 12197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:16.877866 12197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:16.879365 12197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:16.879462 12197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:16.879556 12197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:16.879618 12197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:16.917027 12197 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:38929
I20260812 06:57:16.917073 12428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:38929 every 8 connection(s)
I20260812 06:57:16.918623 12197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:57:16.918965 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 12197
I20260812 06:57:16.919167 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:57:16.919788 27528 external_mini_cluster.cc:1428] Setting key 248f7c30d9a8ff71d5c23f90eb757f00
I20260812 06:57:16.927934 11975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:16.921868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:54569 (local address 127.26.226.62:33119)
0812 06:57:16.922080 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:16.922086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:16.923699 (+ 1613us) server_negotiation.cc:407] Connection header received
0812 06:57:16.925583 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:16.925589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:16.925660 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:16.925808 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:16.927104 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:16.927110 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:16.927115 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:16.927134 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:16.927176 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:16.927188 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:16.927329 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:16.927546 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 06:57:16.927620 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:57:16.928866 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:16.922478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33119 (local address 127.26.226.3:54569)
0812 06:57:16.923557 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:16.923595 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:16.925314 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:16.926037 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:16.926045 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:16.926138 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:16.926802 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:57:16.926805 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:16.926843 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:57:16.926932 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:16.927311 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:16.927322 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:16.927330 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 06:57:16.927618 (+ 288us) client_negotiation.cc:240] Negotiation successful
0812 06:57:16.927887 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":345,"threads_started":1}
I20260812 06:57:16.930135 12430 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33119
I20260812 06:57:16.930442 12430 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:16.931043 12430 heartbeater.cc:511] Master 127.26.226.62:33119 requested a full tablet report, sending...
I20260812 06:57:16.932523 11644 ts_manager.cc:194] Registered new tserver with Master: 5306c30147464628b47c005c6f166306 (127.26.226.3:38929)
I20260812 06:57:16.935732 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 06:57:16.942237 11975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:16.940679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34420 (local address 127.26.226.62:33119)
0812 06:57:16.940848 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:16.940853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:16.941405 (+ 552us) server_negotiation.cc:407] Connection header received
0812 06:57:16.941547 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:16.941551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:16.941602 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:16.941685 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:16.941971 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34420: BlockingRecv error: recv got EOF from 127.0.0.1:34420 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:57:16.943114 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 11739
I20260812 06:57:16.946736 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:16.944221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.26.226.1:34603)
0812 06:57:16.944734 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:16.944740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:16.944765 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:57:16.944822 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:16.944827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:16.945013 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:16.945135 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:16.946100 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:16.946107 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:16.946118 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:16.946138 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:16.946181 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:16.946196 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:16.946303 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:16.946391 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 06:57:16.946470 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":268,"threads_started":1}
I20260812 06:57:16.947973 11964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:17.031492 11970 heartbeater.cc:503] Master 127.26.226.62:33119 was elected leader, sending a full tablet report...
I20260812 06:57:17.032761 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 11739
W20260812 06:57:17.059818 11618 connection.cc:535] server connection from 127.26.226.1:49983 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:57:17.060245 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 11977
I20260812 06:57:17.064252 12452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:17.061185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43980 (local address 127.26.226.2:34283)
0812 06:57:17.061718 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:17.061734 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:17.061761 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:57:17.061827 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:17.061834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:17.062049 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:17.062248 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:17.063333 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:17.063339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:17.063351 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:17.063364 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:17.063411 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:17.063425 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:17.063580 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:17.063705 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 06:57:17.063825 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":246,"threads_started":1}
I20260812 06:57:17.065554 12184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:17.147522 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 11977
I20260812 06:57:17.178157 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 12197
I20260812 06:57:17.182236 12454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:17.179188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.26.226.3:38929)
0812 06:57:17.179872 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:17.179881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:17.179912 (+ 31us) server_negotiation.cc:407] Connection header received
0812 06:57:17.179978 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:17.179984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:17.180304 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:17.180440 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:17.181490 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:17.181497 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:17.181508 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:57:17.181525 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:17.181564 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:57:17.181578 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:17.181715 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:17.181767 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 06:57:17.181887 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":359,"threads_started":1}
I20260812 06:57:17.183470 12413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:17.265837 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 12197
I20260812 06:57:17.295130 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 11589
I20260812 06:57:17.296559 11689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:17.402256 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 11589
2026-08-12T06:57:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2551 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:57:17 dist-test-slave-t3q3 krb5kdc[12465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:57:17 dist-test-slave-t3q3 krb5kdc[12465](info): set up 2 sockets
Aug 12 06:57:17 dist-test-slave-t3q3 krb5kdc[12465](info): commencing operation
krb5kdc: starting...
W20260812 06:57:18.872354 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.422s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:57:19 dist-test-slave-t3q3 krb5kdc[12465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:57:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:57:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:57:19.182250 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:36839
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:36839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:19.529202 12495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:19.529505 12495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:19.529696 12495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:19.536988 12495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:19.537133 12495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:19.537240 12495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:19.537336 12495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:19.537420 12495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:19.547465 12495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:36839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:36839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:19.548957 12495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:19.550325 12495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:19.562371 12507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:19.562348 12502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:19.562439 12501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:19.563791 12495 server_base.cc:1034] running on GCE node
I20260812 06:57:19.564988 12495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:19.566763 12495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:19.568028 12495 hybrid_clock.cc:648] HybridClock initialized: now 1786517839567946 us; error 95 us; skew 500 ppm
I20260812 06:57:19.568442 12495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:57:19 dist-test-slave-t3q3 krb5kdc[12465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517839, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:19.572165 12495 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:57:19.573791 12495 webserver.cc:460] Webserver started at http://127.26.226.62:33997/ using document root <none> and password file <none>
I20260812 06:57:19.574518 12495 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:19.574676 12495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:19.575023 12495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:19.578181 12495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "da15126dea26410a85f33ffbd38417d7"
format_stamp: "Formatted at 2026-08-12 06:57:19 on dist-test-slave-t3q3"
server_key: "710a9bb13b2d2c9de9598bc481ecf97e"
server_key_iv: "ef7e7c5c251a0ce14faeefe3ea326f98"
server_key_version: "encryptionkey@0"
I20260812 06:57:19.578987 12495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "da15126dea26410a85f33ffbd38417d7"
format_stamp: "Formatted at 2026-08-12 06:57:19 on dist-test-slave-t3q3"
server_key: "710a9bb13b2d2c9de9598bc481ecf97e"
server_key_iv: "ef7e7c5c251a0ce14faeefe3ea326f98"
server_key_version: "encryptionkey@0"
I20260812 06:57:19.584532 12495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 06:57:19.588260 12525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:19.589629 12495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:57:19.589825 12495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "da15126dea26410a85f33ffbd38417d7"
format_stamp: "Formatted at 2026-08-12 06:57:19 on dist-test-slave-t3q3"
server_key: "710a9bb13b2d2c9de9598bc481ecf97e"
server_key_iv: "ef7e7c5c251a0ce14faeefe3ea326f98"
server_key_version: "encryptionkey@0"
I20260812 06:57:19.590029 12495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:19.619122 12495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:19.623968 12495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:19.624338 12495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:19.643668 12495 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:36839
I20260812 06:57:19.643713 12625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:36839 every 8 connection(s)
I20260812 06:57:19.645085 12495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:19.649223 12626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:19.653640 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 12495
I20260812 06:57:19.653894 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:57:19.654716 27528 external_mini_cluster.cc:1428] Setting key 5b20b19b110706b7c373a1eeabc6d354
I20260812 06:57:19.661535 12626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7: Bootstrap starting.
I20260812 06:57:19.665405 12626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:19.666675 12626 log.cc:824] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:19.670483 12626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7: No bootstrap required, opened a new log
Aug 12 06:57:19 dist-test-slave-t3q3 krb5kdc[12465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:57:19.681710 12631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:19.656882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35214 (local address 127.26.226.62:36839)
0812 06:57:19.657650 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:19.657662 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:19.657707 (+ 45us) server_negotiation.cc:407] Connection header received
0812 06:57:19.659807 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:19.659832 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:19.660321 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:19.660664 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:19.662703 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:19.664181 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:19.666114 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:19.666939 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:19.667122 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:19.672152 (+ 5030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:19.672165 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:19.672191 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:19.672250 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:19.676628 (+ 4378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:19.677464 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:19.677473 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:19.677482 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:19.677708 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:19.678085 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:19.678090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:19.678093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:19.678904 (+ 811us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:19.679245 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:19.679659 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 06:57:19.680214 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":352,"threads_started":1}
I20260812 06:57:19.681825 12626 raft_consensus.cc:348] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da15126dea26410a85f33ffbd38417d7" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36839 } }
I20260812 06:57:19.682303 12626 raft_consensus.cc:374] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:19.682439 12626 raft_consensus.cc:729] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da15126dea26410a85f33ffbd38417d7, State: Initialized, Role: FOLLOWER
I20260812 06:57:19.683055 12626 consensus_queue.cc:260] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da15126dea26410a85f33ffbd38417d7" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36839 } }
I20260812 06:57:19.683298 12626 raft_consensus.cc:388] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:57:19.683449 12626 raft_consensus.cc:482] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:57:19.683657 12626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:19.686079 12626 raft_consensus.cc:504] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da15126dea26410a85f33ffbd38417d7" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36839 } }
I20260812 06:57:19.686635 12626 leader_election.cc:302] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da15126dea26410a85f33ffbd38417d7; no voters:
I20260812 06:57:19.687233 12626 leader_election.cc:288] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:57:19.687388 12635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:19.688107 12635 raft_consensus.cc:686] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [term 1 LEADER]: Becoming Leader. State: Replica: da15126dea26410a85f33ffbd38417d7, State: Running, Role: LEADER
I20260812 06:57:19.688673 12635 consensus_queue.cc:237] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da15126dea26410a85f33ffbd38417d7" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36839 } }
I20260812 06:57:19.689424 12626 sys_catalog.cc:564] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:19.691617 12642 sys_catalog.cc:455] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da15126dea26410a85f33ffbd38417d7. Latest consensus state: current_term: 1 leader_uuid: "da15126dea26410a85f33ffbd38417d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da15126dea26410a85f33ffbd38417d7" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36839 } } }
I20260812 06:57:19.692106 12642 sys_catalog.cc:458] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:19.692740 12636 sys_catalog.cc:455] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da15126dea26410a85f33ffbd38417d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da15126dea26410a85f33ffbd38417d7" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 36839 } } }
I20260812 06:57:19.693140 12636 sys_catalog.cc:458] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:19.693145 12648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:19.698866 12648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:19.708577 12648 catalog_manager.cc:1347] Generated new cluster ID: 11b94a3dd67c46aa8f15c3109fd39178
I20260812 06:57:19.708719 12648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:19.720175 12648 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:57:19.721539 12648 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:19.736158 12648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7: Generated new TSK 0
I20260812 06:57:19.737093 12648 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:57:19.825551 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36839
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:20.192466 12675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:20.192776 12675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:20.192879 12675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:20.200755 12675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:20.200919 12675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:20.201102 12675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:57:20.211261 12675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36839
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:20.212649 12675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:20.213946 12675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:20.227663 12690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:20.227741 12688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:20.227711 12693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:20.228663 12675 server_base.cc:1034] running on GCE node
I20260812 06:57:20.229292 12675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:20.230226 12675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:20.231460 12675 hybrid_clock.cc:648] HybridClock initialized: now 1786517840231376 us; error 68 us; skew 500 ppm
I20260812 06:57:20.231837 12675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:57:20 dist-test-slave-t3q3 krb5kdc[12465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517840, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:20.235262 12675 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:57:20.236927 12675 webserver.cc:460] Webserver started at http://127.26.226.1:45935/ using document root <none> and password file <none>
I20260812 06:57:20.237660 12675 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:20.237768 12675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:20.238030 12675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:20.241080 12675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "b5cbe3f356684932866794a04f3c388f"
format_stamp: "Formatted at 2026-08-12 06:57:20 on dist-test-slave-t3q3"
server_key: "d97612cd5a27efd90f240c3a3e42b140"
server_key_iv: "d895fe7bbe4157b52a9b0d5430acd4c2"
server_key_version: "encryptionkey@0"
I20260812 06:57:20.241830 12675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "b5cbe3f356684932866794a04f3c388f"
format_stamp: "Formatted at 2026-08-12 06:57:20 on dist-test-slave-t3q3"
server_key: "d97612cd5a27efd90f240c3a3e42b140"
server_key_iv: "d895fe7bbe4157b52a9b0d5430acd4c2"
server_key_version: "encryptionkey@0"
I20260812 06:57:20.247041 12675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:57:20.250697 12706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:20.252310 12675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:20.252516 12675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "b5cbe3f356684932866794a04f3c388f"
format_stamp: "Formatted at 2026-08-12 06:57:20 on dist-test-slave-t3q3"
server_key: "d97612cd5a27efd90f240c3a3e42b140"
server_key_iv: "d895fe7bbe4157b52a9b0d5430acd4c2"
server_key_version: "encryptionkey@0"
I20260812 06:57:20.252754 12675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:20.270373 12675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:20.274669 12675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:20.274983 12675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:20.275983 12675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:20.277483 12675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:20.277594 12675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:20.277755 12675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:20.277830 12675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:20.313287 12675 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:44125
I20260812 06:57:20.313313 12905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:44125 every 8 connection(s)
I20260812 06:57:20.314623 12675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:57:20.323741 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 12675
I20260812 06:57:20.323987 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:57:20.324852 27528 external_mini_cluster.cc:1428] Setting key f35c38e7700dc5f3250e261014689b6a
Aug 12 06:57:20 dist-test-slave-t3q3 krb5kdc[12465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517840, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:57:20.336839 12913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:20.317257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:41975 (local address 127.26.226.62:36839)
0812 06:57:20.317778 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:20.317785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:20.318740 (+ 955us) server_negotiation.cc:407] Connection header received
0812 06:57:20.320620 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:20.320625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:20.320716 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:20.320810 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:20.322263 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:20.322970 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:20.325526 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:20.326051 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:20.326154 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:20.329847 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:20.329856 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:20.329863 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:20.329909 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:20.333506 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:20.334256 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:20.334264 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:20.334270 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:20.334430 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:20.334853 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:20.334860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:20.334866 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:20.335237 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:20.335405 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:20.336057 (+ 652us) server_negotiation.cc:299] Negotiation successful
0812 06:57:20.336346 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":239,"threads_started":1}
I20260812 06:57:20.337495 12918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:20.317815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36839 (local address 127.26.226.1:41975)
0812 06:57:20.318565 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:20.318611 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:20.320355 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:20.321043 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:20.321050 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:20.321550 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:20.322110 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:20.322128 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:20.323203 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:20.323212 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:20.325343 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:20.325358 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:20.326290 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:20.326298 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:20.326581 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:20.327322 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:20.327344 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:20.329619 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:20.333661 (+ 4042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:20.333668 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:20.333680 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:20.334108 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:20.334539 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:20.334542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:20.334545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:20.334732 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:20.335359 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:20.335365 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:20.335674 (+ 309us) client_negotiation.cc:769] Sending connection context
0812 06:57:20.335890 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 06:57:20.336367 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":291,"threads_started":1}
I20260812 06:57:20.339052 12908 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36839
I20260812 06:57:20.339385 12908 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:20.340327 12908 heartbeater.cc:511] Master 127.26.226.62:36839 requested a full tablet report, sending...
I20260812 06:57:20.342610 12561 ts_manager.cc:194] Registered new tserver with Master: b5cbe3f356684932866794a04f3c388f (127.26.226.1:44125)
I20260812 06:57:20.344832 12561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:41975
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:57:20.393411 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36839
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:20.742249 12927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:20.742534 12927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:20.742720 12927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:20.750283 12927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:20.750397 12927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:20.750593 12927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:57:20.761320 12927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36839
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:20.762686 12927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:20.764003 12927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:20.777272 12936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:20.777330 12940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:20.777333 12937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:20.778671 12927 server_base.cc:1034] running on GCE node
I20260812 06:57:20.779383 12927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:20.780400 12927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:20.781633 12927 hybrid_clock.cc:648] HybridClock initialized: now 1786517840781565 us; error 58 us; skew 500 ppm
I20260812 06:57:20.781991 12927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:57:20 dist-test-slave-t3q3 krb5kdc[12465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517840, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:20.785989 12927 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:57:20.787559 12927 webserver.cc:460] Webserver started at http://127.26.226.2:44331/ using document root <none> and password file <none>
I20260812 06:57:20.788393 12927 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:20.788549 12927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:20.788863 12927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:20.791868 12927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "bcb73c15cd0944d3b58dbd2d01653a3f"
format_stamp: "Formatted at 2026-08-12 06:57:20 on dist-test-slave-t3q3"
server_key: "b4e9b954d3c2a10a527db020b0a813f2"
server_key_iv: "290b92e571728edf924b60cca2191855"
server_key_version: "encryptionkey@0"
I20260812 06:57:20.792773 12927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "bcb73c15cd0944d3b58dbd2d01653a3f"
format_stamp: "Formatted at 2026-08-12 06:57:20 on dist-test-slave-t3q3"
server_key: "b4e9b954d3c2a10a527db020b0a813f2"
server_key_iv: "290b92e571728edf924b60cca2191855"
server_key_version: "encryptionkey@0"
I20260812 06:57:20.798599 12927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 06:57:20.802325 12949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:20.803709 12927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:57:20.803898 12927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "bcb73c15cd0944d3b58dbd2d01653a3f"
format_stamp: "Formatted at 2026-08-12 06:57:20 on dist-test-slave-t3q3"
server_key: "b4e9b954d3c2a10a527db020b0a813f2"
server_key_iv: "290b92e571728edf924b60cca2191855"
server_key_version: "encryptionkey@0"
I20260812 06:57:20.804157 12927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:20.842837 12927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:20.847018 12927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:20.847355 12927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:20.848407 12927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:20.849831 12927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:20.849937 12927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:20.850093 12927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:20.850167 12927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:20.885183 12927 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:39093
I20260812 06:57:20.885279 13157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:39093 every 8 connection(s)
I20260812 06:57:20.886521 12927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:57:20.896812 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 12927
I20260812 06:57:20.897047 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:57:20.897495 27528 external_mini_cluster.cc:1428] Setting key 9ec3937ef9e88b2078579a0a9a8239d8
Aug 12 06:57:20 dist-test-slave-t3q3 krb5kdc[12465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517840, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:57:20.910979 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:20.889298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:43089 (local address 127.26.226.62:36839)
0812 06:57:20.889796 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:20.889802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:20.890807 (+ 1005us) server_negotiation.cc:407] Connection header received
0812 06:57:20.893032 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:20.893037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:20.893139 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:20.893410 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:20.895437 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:20.896315 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:20.898239 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:20.898881 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:20.899026 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:20.903496 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:20.903513 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:20.903520 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:20.903571 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:20.907223 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:20.908172 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:20.908180 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:20.908186 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:20.908358 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:20.908845 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:20.908852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:20.908858 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:20.909275 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:20.909443 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:20.910187 (+ 744us) server_negotiation.cc:299] Negotiation successful
0812 06:57:20.910454 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":223,"threads_started":1}
I20260812 06:57:20.911700 13164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:20.889898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36839 (local address 127.26.226.2:43089)
0812 06:57:20.890632 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:20.890672 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:20.892570 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:20.893641 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:20.893649 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:20.894347 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:20.895194 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:20.895224 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:20.896486 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:20.896490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:20.898011 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:20.898032 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:20.899133 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:20.899139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:20.899417 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:20.900296 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:20.900328 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:20.903225 (+ 2897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:20.907437 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:20.907445 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:20.907461 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:20.908013 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:20.908492 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:20.908497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:20.908501 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:20.908715 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:20.909429 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:20.909436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:20.909780 (+ 344us) client_negotiation.cc:769] Sending connection context
0812 06:57:20.910038 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 06:57:20.910489 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":302,"threads_started":1}
I20260812 06:57:20.913420 13159 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36839
I20260812 06:57:20.913826 13159 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:20.914834 13159 heartbeater.cc:511] Master 127.26.226.62:36839 requested a full tablet report, sending...
I20260812 06:57:20.916858 12561 ts_manager.cc:194] Registered new tserver with Master: bcb73c15cd0944d3b58dbd2d01653a3f (127.26.226.2:39093)
I20260812 06:57:20.917853 12561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:43089
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:57:20.969619 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36839
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:21.329294 13170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:21.329586 13170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:21.329685 13170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:21.337188 13170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:21.337302 13170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:21.337417 13170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:57:21.347159 13170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:36839
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:57:21.348022 12908 heartbeater.cc:503] Master 127.26.226.62:36839 was elected leader, sending a full tablet report...
I20260812 06:57:21.348623 13170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:21.350399 13170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:21.364215 13177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:21.364264 13178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:21.364292 13181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:21.365738 13170 server_base.cc:1034] running on GCE node
I20260812 06:57:21.366380 13170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:21.367347 13170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:21.368579 13170 hybrid_clock.cc:648] HybridClock initialized: now 1786517841368536 us; error 32 us; skew 500 ppm
I20260812 06:57:21.368896 13170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:57:21 dist-test-slave-t3q3 krb5kdc[12465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517841, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:21.372531 13170 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:57:21.374056 13170 webserver.cc:460] Webserver started at http://127.26.226.3:43687/ using document root <none> and password file <none>
I20260812 06:57:21.374782 13170 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:21.374889 13170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:21.375162 13170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:21.378192 13170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "0919bd56cb684fe192a46f7d009bdda7"
format_stamp: "Formatted at 2026-08-12 06:57:21 on dist-test-slave-t3q3"
server_key: "535a44d5432a9e78ff2ec0aa654b4715"
server_key_iv: "f2afd7f60879e0757ab67e72ed7ddc85"
server_key_version: "encryptionkey@0"
I20260812 06:57:21.378938 13170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "0919bd56cb684fe192a46f7d009bdda7"
format_stamp: "Formatted at 2026-08-12 06:57:21 on dist-test-slave-t3q3"
server_key: "535a44d5432a9e78ff2ec0aa654b4715"
server_key_iv: "f2afd7f60879e0757ab67e72ed7ddc85"
server_key_version: "encryptionkey@0"
I20260812 06:57:21.384506 13170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 06:57:21.387946 13191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:21.389519 13170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:57:21.389652 13170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "0919bd56cb684fe192a46f7d009bdda7"
format_stamp: "Formatted at 2026-08-12 06:57:21 on dist-test-slave-t3q3"
server_key: "535a44d5432a9e78ff2ec0aa654b4715"
server_key_iv: "f2afd7f60879e0757ab67e72ed7ddc85"
server_key_version: "encryptionkey@0"
I20260812 06:57:21.389817 13170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:21.403308 13170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:21.407693 13170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:21.407972 13170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:21.409103 13170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:21.410485 13170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:21.410560 13170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:21.410645 13170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:21.410688 13170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:21.445474 13170 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:37923
I20260812 06:57:21.445523 13376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:37923 every 8 connection(s)
I20260812 06:57:21.446866 13170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:57:21.452838 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 13170
I20260812 06:57:21.453078 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:57:21.453518 27528 external_mini_cluster.cc:1428] Setting key 79706eff6900b452d504ea804f616d3f
Aug 12 06:57:21 dist-test-slave-t3q3 krb5kdc[12465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517841, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:57:21.468303 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.449660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:43771 (local address 127.26.226.62:36839)
0812 06:57:21.450128 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:21.450135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:21.451221 (+ 1086us) server_negotiation.cc:407] Connection header received
0812 06:57:21.453230 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:21.453241 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:21.453316 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:21.453442 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:21.455447 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.456594 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.458342 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.458727 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.458825 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.462298 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:21.462303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:21.462306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:21.462358 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:21.465016 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:21.465770 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:21.465774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:21.465776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:21.465904 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:21.466403 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:21.466407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:21.466409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:21.466663 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:21.466796 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:21.467610 (+ 814us) server_negotiation.cc:299] Negotiation successful
0812 06:57:21.467871 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":211,"threads_started":1}
I20260812 06:57:21.469449 13383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.450341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:36839 (local address 127.26.226.3:43771)
0812 06:57:21.451049 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:21.451088 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:21.452819 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:21.453733 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:21.453741 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:21.454437 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:21.455238 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.455257 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.456763 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.456771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.458192 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.458210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.458924 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.458929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.459129 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.459766 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:21.459789 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:21.462076 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:21.465161 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:21.465169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:21.465182 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:21.465660 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:21.466080 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:21.466084 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:21.466086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:21.466297 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:21.466786 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:21.466792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:21.467144 (+ 352us) client_negotiation.cc:769] Sending connection context
0812 06:57:21.467467 (+ 323us) client_negotiation.cc:240] Negotiation successful
0812 06:57:21.468008 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":290,"threads_started":1}
I20260812 06:57:21.470814 13378 heartbeater.cc:347] Connected to a master server at 127.26.226.62:36839
I20260812 06:57:21.471155 13378 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:21.471964 13378 heartbeater.cc:511] Master 127.26.226.62:36839 requested a full tablet report, sending...
I20260812 06:57:21.473506 12561 ts_manager.cc:194] Registered new tserver with Master: 0919bd56cb684fe192a46f7d009bdda7 (127.26.226.3:37923)
I20260812 06:57:21.474301 12561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:43771
I20260812 06:57:21.482167 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:57:21.499733 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.486840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35228 (local address 127.26.226.62:36839)
0812 06:57:21.487079 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:21.487088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:21.487376 (+ 288us) server_negotiation.cc:407] Connection header received
0812 06:57:21.487569 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:21.487573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:21.487641 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:21.487741 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:21.489250 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.490145 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.491491 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.491911 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.492024 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.494461 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:21.494473 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:21.494495 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:21.494540 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:21.497317 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:21.497936 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:21.497940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:21.497942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:21.498095 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:21.498428 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:21.498431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:21.498433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:21.498741 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:21.498931 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:21.499217 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 06:57:21.499379 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 06:57:21.504561 12561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:57:21.511659 12561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:57:21.542246 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.523973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:44125 (local address 127.0.0.1:55304)
0812 06:57:21.527288 (+ 3315us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:21.527335 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:21.528982 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:21.530312 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:21.530323 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:21.530365 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:21.530892 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.530909 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.534968 (+ 4059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.534977 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.539138 (+ 4161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.539151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.541235 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.541242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.541450 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.541501 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 06:57:21.541645 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 06:57:21.541857 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3086,"thread_start_us":2932,"threads_started":1}
I20260812 06:57:21.542369 13400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.524357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55304 (local address 127.26.226.1:44125)
0812 06:57:21.526715 (+ 2358us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:21.526727 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:21.527485 (+ 758us) server_negotiation.cc:407] Connection header received
0812 06:57:21.529305 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:21.529318 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:21.529643 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:21.529853 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:21.531181 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.533376 (+ 2195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.539332 (+ 5956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.540990 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.541143 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.541377 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:21.541712 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 06:57:21.541895 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2134,"thread_start_us":1932,"threads_started":1}
I20260812 06:57:21.546857 13399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.524166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:37923 (local address 127.0.0.1:56308)
0812 06:57:21.538285 (+ 14119us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:21.538310 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:21.538742 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:21.539764 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:21.539768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:21.539797 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:21.540174 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.540187 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.542120 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.542125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.544456 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.544471 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.546005 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.546011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.546189 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.546221 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:57:21.546338 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 06:57:21.546475 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13962,"thread_start_us":13796,"threads_started":1}
I20260812 06:57:21.546908 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.524671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56308 (local address 127.26.226.3:37923)
0812 06:57:21.526045 (+ 1374us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:21.526059 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:21.538445 (+ 12386us) server_negotiation.cc:407] Connection header received
0812 06:57:21.538947 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:21.538956 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:21.539240 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:21.539432 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:21.540385 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.541910 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.544640 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.545789 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.545955 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.546216 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:21.546409 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 06:57:21.546571 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1138,"thread_start_us":895,"threads_started":1}
I20260812 06:57:21.547981 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.527569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:39093 (local address 127.0.0.1:53206)
0812 06:57:21.538574 (+ 11005us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:21.538591 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:21.538714 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:21.540251 (+ 1537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:21.540255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:21.540279 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:21.540624 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.540633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.542518 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.542523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.545039 (+ 2516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.545054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.547185 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.547192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.547374 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.547405 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:57:21.547510 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 06:57:21.547644 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10880,"thread_start_us":10882,"threads_started":1}
I20260812 06:57:21.547991 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.527742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53206 (local address 127.26.226.2:39093)
0812 06:57:21.539738 (+ 11996us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:21.539745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:21.539768 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:57:21.539842 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:21.539847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:21.539994 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:21.540235 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:21.540818 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.542332 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.545235 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.546661 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.546816 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.547019 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:21.547569 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 06:57:21.547709 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11822,"thread_start_us":11699,"threads_started":1}
I20260812 06:57:21.546756 12788 tablet_service.cc:1467] Processing CreateTablet for tablet 5fe8007bfc8e4f4f9e80feab1710bcd4 (DEFAULT_TABLE table=test-table [id=93b0967d444c4fa1aa49baa855572d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:21.549757 12788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fe8007bfc8e4f4f9e80feab1710bcd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:21.550678 13267 tablet_service.cc:1467] Processing CreateTablet for tablet 5fe8007bfc8e4f4f9e80feab1710bcd4 (DEFAULT_TABLE table=test-table [id=93b0967d444c4fa1aa49baa855572d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:21.552039 13267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fe8007bfc8e4f4f9e80feab1710bcd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:21.551577 13031 tablet_service.cc:1467] Processing CreateTablet for tablet 5fe8007bfc8e4f4f9e80feab1710bcd4 (DEFAULT_TABLE table=test-table [id=93b0967d444c4fa1aa49baa855572d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:21.552938 13031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fe8007bfc8e4f4f9e80feab1710bcd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:21.563788 13406 tablet_bootstrap.cc:492] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: Bootstrap starting.
I20260812 06:57:21.565968 13408 tablet_bootstrap.cc:492] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: Bootstrap starting.
I20260812 06:57:21.567667 13412 tablet_bootstrap.cc:492] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: Bootstrap starting.
I20260812 06:57:21.569022 13406 tablet_bootstrap.cc:654] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:21.569622 13408 tablet_bootstrap.cc:654] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:21.570381 13406 log.cc:824] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:21.570678 13408 log.cc:824] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:21.571327 13412 tablet_bootstrap.cc:654] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:21.572559 13412 log.cc:824] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:21.573962 13408 tablet_bootstrap.cc:492] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: No bootstrap required, opened a new log
I20260812 06:57:21.573962 13406 tablet_bootstrap.cc:492] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: No bootstrap required, opened a new log
I20260812 06:57:21.574257 13408 ts_tablet_manager.cc:1397] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 06:57:21.574255 13406 ts_tablet_manager.cc:1397] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260812 06:57:21.575240 13412 tablet_bootstrap.cc:492] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: No bootstrap required, opened a new log
I20260812 06:57:21.575506 13412 ts_tablet_manager.cc:1397] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 06:57:21.582870 13412 raft_consensus.cc:348] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5cbe3f356684932866794a04f3c388f" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 44125 } } peers { permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 } } peers { permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 } }
I20260812 06:57:21.583313 13412 raft_consensus.cc:374] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:21.583431 13412 raft_consensus.cc:729] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb73c15cd0944d3b58dbd2d01653a3f, State: Initialized, Role: FOLLOWER
I20260812 06:57:21.584297 13412 consensus_queue.cc:260] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5cbe3f356684932866794a04f3c388f" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 44125 } } peers { permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 } } peers { permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 } }
I20260812 06:57:21.585615 13159 heartbeater.cc:503] Master 127.26.226.62:36839 was elected leader, sending a full tablet report...
I20260812 06:57:21.585402 13408 raft_consensus.cc:348] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5cbe3f356684932866794a04f3c388f" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 44125 } } peers { permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 } } peers { permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 } }
I20260812 06:57:21.585420 13406 raft_consensus.cc:348] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5cbe3f356684932866794a04f3c388f" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 44125 } } peers { permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 } } peers { permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 } }
I20260812 06:57:21.585863 13408 raft_consensus.cc:374] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:21.585867 13406 raft_consensus.cc:374] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:21.585963 13408 raft_consensus.cc:729] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5cbe3f356684932866794a04f3c388f, State: Initialized, Role: FOLLOWER
I20260812 06:57:21.585963 13406 raft_consensus.cc:729] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0919bd56cb684fe192a46f7d009bdda7, State: Initialized, Role: FOLLOWER
I20260812 06:57:21.586519 13412 ts_tablet_manager.cc:1428] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260812 06:57:21.586582 13408 consensus_queue.cc:260] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5cbe3f356684932866794a04f3c388f" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 44125 } } peers { permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 } } peers { permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 } }
I20260812 06:57:21.586597 13406 consensus_queue.cc:260] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5cbe3f356684932866794a04f3c388f" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 44125 } } peers { permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 } } peers { permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 } }
I20260812 06:57:21.588871 13378 heartbeater.cc:503] Master 127.26.226.62:36839 was elected leader, sending a full tablet report...
I20260812 06:57:21.590189 13408 ts_tablet_manager.cc:1428] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260812 06:57:21.591378 13406 ts_tablet_manager.cc:1428] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: Time spent starting tablet: real 0.017s user 0.010s sys 0.006s
W20260812 06:57:21.638705 13161 tablet.cc:2369] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:57:21.698683 13379 tablet.cc:2369] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:57:21.797613 13429 raft_consensus.cc:482] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:57:21.797960 13429 raft_consensus.cc:504] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5cbe3f356684932866794a04f3c388f" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 44125 } } peers { permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 } } peers { permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 } }
I20260812 06:57:21.799810 13429 leader_election.cc:288] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0919bd56cb684fe192a46f7d009bdda7 (127.26.226.3:37923), bcb73c15cd0944d3b58dbd2d01653a3f (127.26.226.2:39093)
I20260812 06:57:21.807120 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.800322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:38597 (local address 127.26.226.2:39093)
0812 06:57:21.800574 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:21.800583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:21.801408 (+ 825us) server_negotiation.cc:407] Connection header received
0812 06:57:21.801596 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:21.801600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:21.801663 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:21.801743 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:21.802429 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.803276 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.805239 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.806097 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.806230 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.806312 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:21.806689 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 06:57:21.806834 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:57:21.807154 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.800385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:60063 (local address 127.26.226.3:37923)
0812 06:57:21.800663 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:21.800671 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:21.801332 (+ 661us) server_negotiation.cc:407] Connection header received
0812 06:57:21.801989 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:21.801997 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:21.802074 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:21.802179 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:21.802770 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.803615 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.805414 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.806220 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.806386 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.806462 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:21.806760 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 06:57:21.806920 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 06:57:21.807196 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.800372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:39093 (local address 127.26.226.1:38597)
0812 06:57:21.801311 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:21.801330 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:21.801444 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:21.801831 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:21.801837 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:21.801869 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:21.802257 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.802267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.803412 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.803418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.805041 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.805053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.806322 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.806328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.806520 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.806544 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:57:21.806623 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 06:57:21.806756 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":430,"threads_started":1}
I20260812 06:57:21.807217 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.800285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:37923 (local address 127.26.226.1:60063)
0812 06:57:21.801206 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:21.801223 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:21.801418 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:21.802213 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:21.802218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:21.802246 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:21.802616 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.802628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.803759 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.803762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.805260 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:21.805275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.806455 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.806459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:21.806573 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.806595 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:57:21.806716 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 06:57:21.806839 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":627,"threads_started":1}
I20260812 06:57:21.808557 13074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe8007bfc8e4f4f9e80feab1710bcd4" candidate_uuid: "b5cbe3f356684932866794a04f3c388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" is_pre_election: true
I20260812 06:57:21.808604 13296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe8007bfc8e4f4f9e80feab1710bcd4" candidate_uuid: "b5cbe3f356684932866794a04f3c388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0919bd56cb684fe192a46f7d009bdda7" is_pre_election: true
I20260812 06:57:21.809144 13074 raft_consensus.cc:2454] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5cbe3f356684932866794a04f3c388f in term 0.
I20260812 06:57:21.809173 13296 raft_consensus.cc:2454] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5cbe3f356684932866794a04f3c388f in term 0.
I20260812 06:57:21.809844 12707 leader_election.cc:302] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5cbe3f356684932866794a04f3c388f, bcb73c15cd0944d3b58dbd2d01653a3f; no voters:
I20260812 06:57:21.810284 13429 raft_consensus.cc:2781] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:57:21.810418 13429 raft_consensus.cc:482] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:57:21.810492 13429 raft_consensus.cc:3037] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:21.812839 13429 raft_consensus.cc:504] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5cbe3f356684932866794a04f3c388f" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 44125 } } peers { permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 } } peers { permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 } }
I20260812 06:57:21.813519 13429 leader_election.cc:288] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [CANDIDATE]: Term 1 election: Requested vote from peers 0919bd56cb684fe192a46f7d009bdda7 (127.26.226.3:37923), bcb73c15cd0944d3b58dbd2d01653a3f (127.26.226.2:39093)
I20260812 06:57:21.813999 13296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe8007bfc8e4f4f9e80feab1710bcd4" candidate_uuid: "b5cbe3f356684932866794a04f3c388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0919bd56cb684fe192a46f7d009bdda7"
I20260812 06:57:21.814091 13074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe8007bfc8e4f4f9e80feab1710bcd4" candidate_uuid: "b5cbe3f356684932866794a04f3c388f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f"
I20260812 06:57:21.814220 13296 raft_consensus.cc:3037] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:21.814348 13074 raft_consensus.cc:3037] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:21.816597 13296 raft_consensus.cc:2454] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5cbe3f356684932866794a04f3c388f in term 1.
I20260812 06:57:21.816704 13074 raft_consensus.cc:2454] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5cbe3f356684932866794a04f3c388f in term 1.
I20260812 06:57:21.817158 12710 leader_election.cc:302] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0919bd56cb684fe192a46f7d009bdda7, b5cbe3f356684932866794a04f3c388f; no voters:
W20260812 06:57:21.817194 12910 tablet.cc:2369] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:57:21.817509 13429 raft_consensus.cc:2781] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:21.818053 13429 raft_consensus.cc:686] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 1 LEADER]: Becoming Leader. State: Replica: b5cbe3f356684932866794a04f3c388f, State: Running, Role: LEADER
I20260812 06:57:21.818518 13429 consensus_queue.cc:237] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5cbe3f356684932866794a04f3c388f" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 44125 } } peers { permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 } } peers { permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 } }
I20260812 06:57:21.823679 12558 catalog_manager.cc:5495] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f reported cstate change: term changed from 0 to 1, leader changed from <none> to b5cbe3f356684932866794a04f3c388f (127.26.226.1). New cstate: current_term: 1 leader_uuid: "b5cbe3f356684932866794a04f3c388f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5cbe3f356684932866794a04f3c388f" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 44125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 } health_report { overall_health: UNKNOWN } } }
I20260812 06:57:21.908999 13400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.903225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.26.226.1:44125)
0812 06:57:21.903454 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:21.903462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:21.903550 (+ 88us) server_negotiation.cc:407] Connection header received
0812 06:57:21.903816 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:21.903821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:21.903926 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:21.904113 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:21.904508 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.905251 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.906609 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.907034 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.907178 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.907767 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:21.908374 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:21.908544 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:21.908613 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 06:57:21.908727 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:57:21.917459 13296 raft_consensus.cc:1261] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7 [term 1 FOLLOWER]: Refusing update from remote peer b5cbe3f356684932866794a04f3c388f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:21.917459 13074 raft_consensus.cc:1261] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f [term 1 FOLLOWER]: Refusing update from remote peer b5cbe3f356684932866794a04f3c388f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:21.918681 13429 consensus_queue.cc:1035] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0919bd56cb684fe192a46f7d009bdda7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:21.919018 13435 consensus_queue.cc:1035] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb73c15cd0944d3b58dbd2d01653a3f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 39093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:21.931245 13438 mvcc.cc:205] Tried to move back new op lower bound from 7317577080483037184 to 7317577080092205056. Current Snapshot: MvccSnapshot[applied={T|T < 7317577080483037184}]
I20260812 06:57:21.936667 13442 mvcc.cc:205] Tried to move back new op lower bound from 7317577080483037184 to 7317577080092205056. Current Snapshot: MvccSnapshot[applied={T|T < 7317577080483037184}]
I20260812 06:57:21.950011 12559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:57:21.950385 12559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:57:21.954358 12559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da15126dea26410a85f33ffbd38417d7: Sending DeleteTablet for 3 replicas of tablet 5fe8007bfc8e4f4f9e80feab1710bcd4
I20260812 06:57:21.955713 13031 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fe8007bfc8e4f4f9e80feab1710bcd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:21 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:53206
I20260812 06:57:21.955699 13267 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fe8007bfc8e4f4f9e80feab1710bcd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:21 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:56308
I20260812 06:57:21.955717 12788 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fe8007bfc8e4f4f9e80feab1710bcd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:21 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:55304
I20260812 06:57:21.961055 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 12675
I20260812 06:57:21.972084 13456 tablet_replica.cc:330] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: stopping tablet replica
I20260812 06:57:21.972786 13456 raft_consensus.cc:2229] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:57:21.975198 13455 tablet_replica.cc:330] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: stopping tablet replica
I20260812 06:57:21.975888 13455 raft_consensus.cc:2229] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:57:21.973825 13456 raft_consensus.cc:2258] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:21.976621 13455 raft_consensus.cc:2258] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:21.977478 13454 tablet_replica.cc:330] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: stopping tablet replica
I20260812 06:57:21.978376 13454 raft_consensus.cc:2229] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:57:21.978935 13454 raft_consensus.cc:2258] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:21.979020 13456 ts_tablet_manager.cc:1905] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:21.979482 13455 ts_tablet_manager.cc:1905] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:21.981199 13454 ts_tablet_manager.cc:1905] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:21.986696 13455 ts_tablet_manager.cc:1918] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:57:21.986892 13455 log.cc:1196] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/5fe8007bfc8e4f4f9e80feab1710bcd4
I20260812 06:57:21.987483 13455 ts_tablet_manager.cc:1939] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P bcb73c15cd0944d3b58dbd2d01653a3f: Deleting consensus metadata
I20260812 06:57:21.987458 13456 ts_tablet_manager.cc:1918] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:57:21.987672 13456 log.cc:1196] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/5fe8007bfc8e4f4f9e80feab1710bcd4
I20260812 06:57:21.988284 13456 ts_tablet_manager.cc:1939] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P b5cbe3f356684932866794a04f3c388f: Deleting consensus metadata
I20260812 06:57:21.989203 13454 ts_tablet_manager.cc:1918] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:57:21.989439 13454 log.cc:1196] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/5fe8007bfc8e4f4f9e80feab1710bcd4
I20260812 06:57:21.990080 13454 ts_tablet_manager.cc:1939] T 5fe8007bfc8e4f4f9e80feab1710bcd4 P 0919bd56cb684fe192a46f7d009bdda7: Deleting consensus metadata
I20260812 06:57:21.990671 12528 catalog_manager.cc:4841] TS b5cbe3f356684932866794a04f3c388f (127.26.226.1:44125): tablet 5fe8007bfc8e4f4f9e80feab1710bcd4 (table test-table [id=93b0967d444c4fa1aa49baa855572d5e]) successfully deleted
Aug 12 06:57:21 dist-test-slave-t3q3 krb5kdc[12465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:57:21.991974 12530 catalog_manager.cc:4841] TS 0919bd56cb684fe192a46f7d009bdda7 (127.26.226.3:37923): tablet 5fe8007bfc8e4f4f9e80feab1710bcd4 (table test-table [id=93b0967d444c4fa1aa49baa855572d5e]) successfully deleted
I20260812 06:57:21.993352 12527 catalog_manager.cc:4841] TS bcb73c15cd0944d3b58dbd2d01653a3f (127.26.226.2:39093): tablet 5fe8007bfc8e4f4f9e80feab1710bcd4 (table test-table [id=93b0967d444c4fa1aa49baa855572d5e]) successfully deleted
I20260812 06:57:22.000664 13400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:21.962952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.26.226.1:44125)
0812 06:57:21.963182 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:21.963191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:21.964939 (+ 1748us) server_negotiation.cc:407] Connection header received
0812 06:57:21.965905 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:21.965914 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:21.966037 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:21.966130 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:21.971122 (+ 4992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.973038 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.975756 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:21.976753 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:21.976869 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:21.992840 (+ 15971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:21.992857 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:21.992877 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:21.992932 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:21.997465 (+ 4533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:21.998246 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:21.998262 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:21.998266 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:21.998476 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:21.998866 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:21.998872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:21.998876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:21.999337 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:21.999632 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:21.999908 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 06:57:22.000227 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:57:22.001469 12898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:22.102015 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 12675
I20260812 06:57:22.128155 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 12927
Aug 12 06:57:22 dist-test-slave-t3q3 krb5kdc[12465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:57:22.143953 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:22.129086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53216 (local address 127.26.226.2:39093)
0812 06:57:22.129287 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:22.129291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:22.129313 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:57:22.129464 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:22.129468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:22.129534 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:22.129619 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:22.130900 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:22.131754 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:22.133195 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:22.133591 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:22.133690 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:22.138015 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:22.138055 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:22.138072 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:22.138123 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:22.141148 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:22.141835 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:22.141844 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:22.141846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:22.142003 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:22.142397 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:22.142400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:22.142402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:22.142808 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:22.143034 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:22.143361 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 06:57:22.143533 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 06:57:22.144845 13149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:22.235941 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 12927
I20260812 06:57:22.263316 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 13170
Aug 12 06:57:22 dist-test-slave-t3q3 krb5kdc[12465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:57:22.279452 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:22.264194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.26.226.3:37923)
0812 06:57:22.264391 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:22.264399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:22.264467 (+ 68us) server_negotiation.cc:407] Connection header received
0812 06:57:22.264714 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:22.264718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:22.264787 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:22.264902 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:22.266260 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:22.266989 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:22.268413 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:22.268820 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:22.268962 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:22.273592 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:22.273609 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:22.273626 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:22.273667 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:22.276494 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:22.277202 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:22.277210 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:22.277213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:22.277361 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:22.277799 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:22.277803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:22.277805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:22.278195 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:22.278536 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:22.278880 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 06:57:22.279115 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:57:22.280390 13368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:22.372277 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 13170
I20260812 06:57:22.400502 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 12495
I20260812 06:57:22.401706 12616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:22.513311 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 12495
2026-08-12T06:57:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5116 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:57:22 dist-test-slave-t3q3 krb5kdc[13471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:57:22 dist-test-slave-t3q3 krb5kdc[13471](info): set up 2 sockets
Aug 12 06:57:22 dist-test-slave-t3q3 krb5kdc[13471](info): commencing operation
krb5kdc: starting...
W20260812 06:57:23.929834 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.363s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:57:24 dist-test-slave-t3q3 krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:57:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:57:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.226.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.226.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/rangeradmin_127.26.226.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/rangeradmin_127.26.226.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.226.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.226.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/rangerlookup_127.26.226.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/rangerlookup_127.26.226.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.226.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.226.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/HTTP_127.26.226.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/HTTP_127.26.226.20@KRBTEST.COM.keytab.
I20260812 06:57:24.373634 27528 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6poUc4/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/postgres -l logfile start
2026-08-12 06:57:26.911 UTC [13532] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 06:57:26.911 UTC [13532] LOG: listening on IPv4 address "127.26.226.20", port 34669
2026-08-12 06:57:26.915 UTC [13532] LOG: listening on Unix socket "/tmp/.s.PGSQL.34669"
2026-08-12 06:57:26.940 UTC [13538] LOG: database system was shut down at 2026-08-12 06:57:25 UTC
2026-08-12 06:57:26.944 UTC [13532] LOG: database system is ready to accept connections
I20260812 06:57:28.183416 27528 mini_postgres.cc:96] Postgres bound to 34669
2026-08-12 06:57:28.212 UTC [13554] FATAL: database "slave" does not exist
127.26.226.20:34669 - accepting connections
I20260812 06:57:28.213333 27528 mini_ranger.cc:161] Starting Ranger...
I20260812 06:57:28.251192 27528 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 06:57:28.758795 27528 mini_ranger.cc:87] Created ranger Postgres database
I20260812 06:57:28.758950 27528 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-admin
2026-08-12 06:57:29,183 [I] DB FLAVOR :POSTGRES
2026-08-12 06:57:29,184 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:57:29,184 [I] Checking connection..
2026-08-12 06:57:29,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 06:57:29,463 [I] Checking connection passed.
2026-08-12 06:57:29,463 [I] --------- Verifying version history table ---------
2026-08-12 06:57:29,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:57:29,728 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:57:29,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:57:30,002 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:57:30,002 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 06:57:30,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 06:57:30.267 UTC [13652] WARNING: there is no transaction in progress
2026-08-12 06:57:30,275 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 06:57:30,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:57:30,540 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 06:57:30,540 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:57:30,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 06:57:30,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 06:57:31,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t3q3.c.gcp-upstream.internal','N') ;"
2026-08-12 06:57:31,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:57:31,593 [I] Table x_portal_user does not exist in database ranger
2026-08-12 06:57:31,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:57:31,870 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 06:57:31,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:57:32,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:57:32,410 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 06:57:32,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 06:57:33.380 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.396 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.412 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.428 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.443 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.464 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.517 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.524 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.538 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.549 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.560 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.574 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.583 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.590 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.596 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:33.603 UTC [13852] WARNING: there is no transaction in progress
2026-08-12 06:57:34,228 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 06:57:34,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 06:57:34,487 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 06:57:34,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:57:34,728 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 06:57:34,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:57:35,002 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 06:57:35,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:57:35,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:57:35,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t3q3.c.gcp-upstream.internal';"
2026-08-12 06:57:35,807 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 06:57:35,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 06:57:36,069 [I] DB_PATCHES have already been applied
I20260812 06:57:36.075891 27528 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-admin:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6poUc4/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 06:57:36.076026 27528 mini_ranger.cc:193] Using host: 127.26.226.20
I20260812 06:57:36.100023 27528 mini_ranger.cc:227] Ranger admin URL: http://127.26.226.20:33019
Aug 12, 2026 6:57:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:57:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:57:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 6:57:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 6:57:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 6:57:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 6:57:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 06:57:37.535372 27528 mini_ranger.cc:160] Time spent starting Ranger: real 9.322s user 0.002s sys 0.074s
Aug 12, 2026 6:57:58 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 6:57:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 6:57:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 6:57:58 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 6:57:59 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 06:57:59.801298 27528 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.226.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.226.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/rangerkms_127.26.226.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/rangerkms_127.26.226.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.226.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.226.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.226.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/HTTP_127.26.226.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.226.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/HTTP_127.26.226.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 06:58:00 dist-test-slave-t3q3 krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517880, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 06:58:00.023859 27528 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 06:58:00.055016 27528 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 06:58:00.933029 27528 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 06:58:00.933254 27528 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-kms
I20260812 06:58:00.933379 27528 mini_ranger_kms.cc:227] Using postgres at 127.26.226.20:34669
2026-08-12 06:58:01,538 [I] DB FLAVOR :POSTGRES
2026-08-12 06:58:01,539 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:58:01,539 [I] Checking connection
2026-08-12 06:58:01,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 06:58:01,793 [I] connection success
2026-08-12 06:58:01,793 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 06:58:01,793 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 06:58:01,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 06:58:02,053 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 06:58:02,054 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:58:02,054 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 06:58:02,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:34669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 06:58:02,335 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 06:58:02.798966 27528 mini_ranger_kms.cc:310] Created kms service
I20260812 06:58:03.003697 27528 mini_ranger_kms.cc:326] Created kudu user
I20260812 06:58:03.054004 27528 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 06:58:03 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517880, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:58:03.499356 27528 mini_ranger_kms.cc:384] Added ranger policy
I20260812 06:58:03.500730 27528 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-kms:/tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 06:58:03.500821 27528 mini_ranger_kms.cc:240] Using host: 127.26.226.20
I20260812 06:58:03.529891 27528 mini_ranger_kms.cc:276] Ranger KMS PID: 14199
I20260812 06:58:03.530166 27528 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.226.20:34657
Aug 12, 2026 6:58:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:58:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:58:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 6:58:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 6:58:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 6:58:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 06:58:06 dist-test-slave-t3q3 krb5kdc[13471](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517886, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.226.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:06.493590 27528 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.470s user 0.132s sys 0.459s
I20260812 06:58:06.493986 27528 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 06:58:06.494279 27528 mini_ranger_kms.cc:401] 127.26.226.20:34657/kms/v1/keys
Aug 12 06:58:06 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517886, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.226.20@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
Aug 12 06:58:06 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517886, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.226.20@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
Aug 12 06:58:09 dist-test-slave-t3q3 krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:58:09.380471 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:46465
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:46465
--ranger_config_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:34657/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:58:09.740664 14278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:09.741011 14278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:09.741170 14278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:09.748538 14278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:58:09.748692 14278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:09.748798 14278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:09.748898 14278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:58:09.748980 14278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:58:09.759418 14278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.226.20:34657/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:46465
--ranger_config_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:46465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:58:09.760810 14278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:09.762107 14278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:09.774451 14289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:09.774487 14290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:09.774542 14294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:09.775460 14278 server_base.cc:1034] running on GCE node
I20260812 06:58:09.776432 14278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:09.777908 14278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:09.779134 14278 hybrid_clock.cc:648] HybridClock initialized: now 1786517889779085 us; error 51 us; skew 500 ppm
I20260812 06:58:09.779493 14278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:58:09 dist-test-slave-t3q3 krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517889, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:09.782850 14278 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:58:09.784508 14278 webserver.cc:460] Webserver started at http://127.26.226.62:36675/ using document root <none> and password file <none>
I20260812 06:58:09.785178 14278 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:09.785323 14278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:09.785641 14278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:58:09 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517889, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:58:09.852414 14278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "ed66c844a6f34194a16ba4bb8c24988f"
format_stamp: "Formatted at 2026-08-12 06:58:09 on dist-test-slave-t3q3"
server_key: "8fc56bd21858f4badd559f2b4cd2f60e"
server_key_iv: "0e38cb8b76a0397b316edd91dd82dd5f"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:09.853315 14278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "ed66c844a6f34194a16ba4bb8c24988f"
format_stamp: "Formatted at 2026-08-12 06:58:09 on dist-test-slave-t3q3"
server_key: "8fc56bd21858f4badd559f2b4cd2f60e"
server_key_iv: "0e38cb8b76a0397b316edd91dd82dd5f"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:09.858978 14278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 06:58:09.880218 14308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:09.881814 14278 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260812 06:58:09.882025 14278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "ed66c844a6f34194a16ba4bb8c24988f"
format_stamp: "Formatted at 2026-08-12 06:58:09 on dist-test-slave-t3q3"
server_key: "8fc56bd21858f4badd559f2b4cd2f60e"
server_key_iv: "0e38cb8b76a0397b316edd91dd82dd5f"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:09.882270 14278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:09.896723 14278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:09.901006 14278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:09.901338 14278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:09.921370 14278 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:46465
I20260812 06:58:09.921424 14391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:46465 every 8 connection(s)
I20260812 06:58:09.922890 14278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:58:09.927093 14392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:09.929483 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 14278
I20260812 06:58:09.929710 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:58:09.930402 27528 external_mini_cluster.cc:1428] Setting key a5ef41f83272de90f77fb50166f8dc24
I20260812 06:58:09.939917 14392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f: Bootstrap starting.
I20260812 06:58:09.943585 14392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f: Neither blocks nor log segments found. Creating new log.
Aug 12 06:58:09 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:58:09.944846 14392 log.cc:824] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:09.948666 14392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f: No bootstrap required, opened a new log
I20260812 06:58:09.953568 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.932607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.26.226.62:46465)
0812 06:58:09.933461 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:09.933473 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:09.933518 (+ 45us) server_negotiation.cc:407] Connection header received
0812 06:58:09.935254 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:09.935278 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:09.935685 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:09.936134 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:09.937603 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.938956 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.940408 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.941107 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.941316 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.945467 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:09.945486 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:09.945516 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:09.945581 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:09.949283 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:09.950067 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:09.950074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:09.950081 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:09.950245 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:09.950670 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:09.950673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:09.950675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:09.951246 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:09.951587 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:09.952043 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 06:58:09.952502 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":390,"threads_started":1}
I20260812 06:58:09.958348 14392 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed66c844a6f34194a16ba4bb8c24988f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46465 } }
I20260812 06:58:09.958778 14392 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:09.958912 14392 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed66c844a6f34194a16ba4bb8c24988f, State: Initialized, Role: FOLLOWER
I20260812 06:58:09.959542 14392 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed66c844a6f34194a16ba4bb8c24988f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46465 } }
I20260812 06:58:09.959822 14392 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:58:09.959990 14392 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:58:09.960210 14392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:58:09.962929 14392 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed66c844a6f34194a16ba4bb8c24988f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46465 } }
I20260812 06:58:09.963526 14392 leader_election.cc:302] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed66c844a6f34194a16ba4bb8c24988f; no voters:
I20260812 06:58:09.964267 14392 leader_election.cc:288] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:58:09.964449 14402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:58:09.965183 14402 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [term 1 LEADER]: Becoming Leader. State: Replica: ed66c844a6f34194a16ba4bb8c24988f, State: Running, Role: LEADER
I20260812 06:58:09.965873 14402 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed66c844a6f34194a16ba4bb8c24988f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46465 } }
I20260812 06:58:09.966871 14392 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:58:09.969264 14405 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed66c844a6f34194a16ba4bb8c24988f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed66c844a6f34194a16ba4bb8c24988f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46465 } } }
I20260812 06:58:09.969192 14407 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed66c844a6f34194a16ba4bb8c24988f. Latest consensus state: current_term: 1 leader_uuid: "ed66c844a6f34194a16ba4bb8c24988f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed66c844a6f34194a16ba4bb8c24988f" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46465 } } }
I20260812 06:58:09.969694 14405 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [sys.catalog]: This master's current role is: LEADER
I20260812 06:58:09.969702 14407 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f [sys.catalog]: This master's current role is: LEADER
I20260812 06:58:09.973551 14415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:58:09.979718 14415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:58:09.980777 14392 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 06:58:10.000834 14415 catalog_manager.cc:1347] Generated new cluster ID: 6aa707405d084ab2b6c3d2c6d3acca8d
I20260812 06:58:10.001006 14415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:58:10.026217 14415 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:58:10.027613 14415 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:58:10.033728 14415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f: Generated new TSK 0
I20260812 06:58:10.034567 14415 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:58:10.876185 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:34657/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46465
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:58:11 dist-test-slave-t3q3 krb5kdc[13471](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 06:58:11.355746 14471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:11.356164 14471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:11.356350 14471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:11.364027 14471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:11.364244 14471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:11.364442 14471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:58:11.375590 14471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.226.20:34657/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:58:11.377285 14471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:11.378674 14471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:11.407702 14491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:11.408574 14490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:11.414793 14493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:11.415072 14471 server_base.cc:1034] running on GCE node
I20260812 06:58:11.415920 14471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:11.418804 14471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:11.420146 14471 hybrid_clock.cc:648] HybridClock initialized: now 1786517891420025 us; error 118 us; skew 500 ppm
I20260812 06:58:11.420625 14471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:58:11 dist-test-slave-t3q3 krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:11.428336 14471 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:58:11.430723 14471 webserver.cc:460] Webserver started at http://127.26.226.1:46475/ using document root <none> and password file <none>
I20260812 06:58:11.431735 14471 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:11.431892 14471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:11.432331 14471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:58:11 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:58:11.452346 14471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "91bd6d5951354767aea850a5a80f4293"
format_stamp: "Formatted at 2026-08-12 06:58:11 on dist-test-slave-t3q3"
server_key: "cc5f288e1a238f4ec12195693a2b4d0a"
server_key_iv: "a8ed3dcabfc7ce4f4385a8fc5ca6b3da"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:11.453656 14471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "91bd6d5951354767aea850a5a80f4293"
format_stamp: "Formatted at 2026-08-12 06:58:11 on dist-test-slave-t3q3"
server_key: "cc5f288e1a238f4ec12195693a2b4d0a"
server_key_iv: "a8ed3dcabfc7ce4f4385a8fc5ca6b3da"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:11.466209 14471 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.004s sys 0.008s
I20260812 06:58:11.504822 14508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:11.507161 14471 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.014s sys 0.003s
I20260812 06:58:11.507483 14471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "91bd6d5951354767aea850a5a80f4293"
format_stamp: "Formatted at 2026-08-12 06:58:11 on dist-test-slave-t3q3"
server_key: "cc5f288e1a238f4ec12195693a2b4d0a"
server_key_iv: "a8ed3dcabfc7ce4f4385a8fc5ca6b3da"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:11.507794 14471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:11.593127 14471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:11.598817 14471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:11.599259 14471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:11.612547 14471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:11.627463 14471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:58:11.627663 14471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 06:58:11.627862 14471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:58:11.627981 14471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:58:11 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:58:12.113061 14706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:40909 every 8 connection(s)
I20260812 06:58:12.113480 14471 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:40909
I20260812 06:58:12.115370 14471 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:58:12.122042 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 14471
I20260812 06:58:12.122285 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:58:12.123217 27528 external_mini_cluster.cc:1428] Setting key e67502a43009a564eb0bbf4310016720
Aug 12 06:58:12 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:58:12.184983 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:12.124103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:52453 (local address 127.26.226.62:46465)
0812 06:58:12.152447 (+ 28344us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:12.152454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:12.152481 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:58:12.155739 (+ 3258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:12.155745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:12.155842 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:12.155927 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:12.168215 (+ 12288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:12.169406 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:12.171903 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:12.172543 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:12.172643 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:12.177451 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:12.177458 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:12.177463 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:12.177504 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:12.181123 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:12.182103 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:12.182108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:12.182112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:12.182276 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:12.182820 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:12.182826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:12.182829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:12.183188 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:12.183303 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:12.184348 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0812 06:58:12.184597 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28208,"thread_start_us":28369,"threads_started":1}
I20260812 06:58:12.189278 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:12.123276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:46465 (local address 127.26.226.1:52453)
0812 06:58:12.150995 (+ 27719us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:12.151079 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:12.154181 (+ 3102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:12.156522 (+ 2341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:12.156534 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:12.157196 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:12.157982 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:12.158016 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:12.169662 (+ 11646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:12.169668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:12.171706 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:12.171734 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:12.173054 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:12.173061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:12.173356 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:12.174235 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:12.174266 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:12.177170 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:12.181325 (+ 4155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:12.181334 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:12.181354 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:12.181942 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:12.182423 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:12.182428 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:12.182431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:12.182686 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:12.183410 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:12.183419 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:12.183869 (+ 450us) client_negotiation.cc:769] Sending connection context
0812 06:58:12.185596 (+ 1727us) client_negotiation.cc:240] Negotiation successful
0812 06:58:12.186173 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27338,"thread_start_us":31114,"threads_started":1}
I20260812 06:58:12.197166 14707 heartbeater.cc:347] Connected to a master server at 127.26.226.62:46465
I20260812 06:58:12.197531 14707 heartbeater.cc:464] Registering TS with master...
I20260812 06:58:12.198474 14707 heartbeater.cc:511] Master 127.26.226.62:46465 requested a full tablet report, sending...
I20260812 06:58:12.201232 14336 ts_manager.cc:194] Registered new tserver with Master: 91bd6d5951354767aea850a5a80f4293 (127.26.226.1:40909)
I20260812 06:58:12.203258 14336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:52453
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Aug 12 06:58:12 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
W20260812 06:58:12.263211 27528 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.26.226.2: real 0.137s user 0.000s sys 0.027s
I20260812 06:58:12.264235 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:34657/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46465
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:58:12.719275 14717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:12.719547 14717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:12.719719 14717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:12.726890 14717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:12.727002 14717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:12.727223 14717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:58:12.741036 14717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.226.20:34657/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:58:12.742376 14717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:12.743670 14717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:12.756891 14742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:12.756982 14744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:12.757014 14747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:12.758337 14717 server_base.cc:1034] running on GCE node
I20260812 06:58:12.758970 14717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:12.759953 14717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:12.761193 14717 hybrid_clock.cc:648] HybridClock initialized: now 1786517892761158 us; error 76 us; skew 500 ppm
I20260812 06:58:12.761514 14717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:58:12 dist-test-slave-t3q3 krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:12.764714 14717 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:58:12.766302 14717 webserver.cc:460] Webserver started at http://127.26.226.2:44103/ using document root <none> and password file <none>
I20260812 06:58:12.767021 14717 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:12.767194 14717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:12.767593 14717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:58:12 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:58:12.784042 14717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "804cece7ba4b45dea8ff385aad7e9a8a"
format_stamp: "Formatted at 2026-08-12 06:58:12 on dist-test-slave-t3q3"
server_key: "f0b2bfabdeb0b9fc48b3b715024140e9"
server_key_iv: "ced6fa37b8d97693dff006fc4f7515f3"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:12.785100 14717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "804cece7ba4b45dea8ff385aad7e9a8a"
format_stamp: "Formatted at 2026-08-12 06:58:12 on dist-test-slave-t3q3"
server_key: "f0b2bfabdeb0b9fc48b3b715024140e9"
server_key_iv: "ced6fa37b8d97693dff006fc4f7515f3"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:12.791050 14717 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 06:58:12.808929 14765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:12.810714 14717 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:58:12.810922 14717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "804cece7ba4b45dea8ff385aad7e9a8a"
format_stamp: "Formatted at 2026-08-12 06:58:12 on dist-test-slave-t3q3"
server_key: "f0b2bfabdeb0b9fc48b3b715024140e9"
server_key_iv: "ced6fa37b8d97693dff006fc4f7515f3"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:12.811192 14717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:12.839815 14717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:12.844321 14717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:12.844677 14717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:12.845678 14717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:12.847179 14717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:58:12.847288 14717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:12.847437 14717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:58:12.847512 14717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:12.882236 14717 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:38203
I20260812 06:58:12.882274 14969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:38203 every 8 connection(s)
I20260812 06:58:12.883562 14717 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:58:12.888293 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 14717
I20260812 06:58:12.888490 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:58:12.889061 27528 external_mini_cluster.cc:1428] Setting key da989581f49a93d662999d3f286b6ac3
Aug 12 06:58:12 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:58:12.907264 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:12.886156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:37335 (local address 127.26.226.62:46465)
0812 06:58:12.886673 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:12.886680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:12.887777 (+ 1097us) server_negotiation.cc:407] Connection header received
0812 06:58:12.890284 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:12.890290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:12.890392 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:12.890531 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:12.892472 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:12.893359 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:12.895932 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:12.896465 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:12.896592 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:12.901139 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:12.901145 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:12.901149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:12.901182 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:12.903935 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:12.904777 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:12.904781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:12.904783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:12.904912 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:12.905432 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:12.905436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:12.905438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:12.905730 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:12.905899 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:12.906619 (+ 720us) server_negotiation.cc:299] Negotiation successful
0812 06:58:12.906860 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":259,"threads_started":1}
I20260812 06:58:12.908305 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:12.886877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:46465 (local address 127.26.226.2:37335)
0812 06:58:12.887612 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:12.887658 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:12.889907 (+ 2249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:12.890738 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:12.890746 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:12.891304 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:12.892203 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:12.892228 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:12.893546 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:12.893554 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:12.895747 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:12.895767 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:12.896722 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:12.896732 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:12.897096 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:12.897881 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:12.897916 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:12.900857 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:12.904120 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:12.904132 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:12.904153 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:12.904613 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:12.905031 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:12.905037 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:12.905040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:12.905253 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:12.905871 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:12.905879 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:12.906224 (+ 345us) client_negotiation.cc:769] Sending connection context
0812 06:58:12.906492 (+ 268us) client_negotiation.cc:240] Negotiation successful
0812 06:58:12.907036 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":297,"threads_started":1}
I20260812 06:58:12.909880 14971 heartbeater.cc:347] Connected to a master server at 127.26.226.62:46465
I20260812 06:58:12.910218 14971 heartbeater.cc:464] Registering TS with master...
I20260812 06:58:12.910965 14971 heartbeater.cc:511] Master 127.26.226.62:46465 requested a full tablet report, sending...
I20260812 06:58:12.912578 14336 ts_manager.cc:194] Registered new tserver with Master: 804cece7ba4b45dea8ff385aad7e9a8a (127.26.226.2:38203)
I20260812 06:58:12.913435 14336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:37335
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:58:12.959602 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:34657/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46465
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:58:13.207160 14707 heartbeater.cc:503] Master 127.26.226.62:46465 was elected leader, sending a full tablet report...
W20260812 06:58:13.339222 14982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:13.339598 14982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:13.339775 14982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:13.348037 14982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:13.348317 14982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:13.348520 14982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:58:13.358971 14982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.226.20:34657/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:58:13.360491 14982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:13.361841 14982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:13.376226 14990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:13.376299 14994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:13.376332 14996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:13.377473 14982 server_base.cc:1034] running on GCE node
I20260812 06:58:13.378238 14982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:13.379274 14982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:13.380558 14982 hybrid_clock.cc:648] HybridClock initialized: now 1786517893380480 us; error 66 us; skew 500 ppm
I20260812 06:58:13.380951 14982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:58:13 dist-test-slave-t3q3 krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:13.384537 14982 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:58:13.386327 14982 webserver.cc:460] Webserver started at http://127.26.226.3:32993/ using document root <none> and password file <none>
I20260812 06:58:13.387167 14982 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:13.387285 14982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:13.387601 14982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:58:13 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:58:13.405047 14982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "7593b83a15d14273a0c5555b628d8dac"
format_stamp: "Formatted at 2026-08-12 06:58:13 on dist-test-slave-t3q3"
server_key: "c6f9c59bff578c1d8e95f5887dfce495"
server_key_iv: "76d65423f1d93897b2a191fb6c9a15fb"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:13.405990 14982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "7593b83a15d14273a0c5555b628d8dac"
format_stamp: "Formatted at 2026-08-12 06:58:13 on dist-test-slave-t3q3"
server_key: "c6f9c59bff578c1d8e95f5887dfce495"
server_key_iv: "76d65423f1d93897b2a191fb6c9a15fb"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:13.412297 14982 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260812 06:58:13.429311 15013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:13.430927 14982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:58:13.431141 14982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "7593b83a15d14273a0c5555b628d8dac"
format_stamp: "Formatted at 2026-08-12 06:58:13 on dist-test-slave-t3q3"
server_key: "c6f9c59bff578c1d8e95f5887dfce495"
server_key_iv: "76d65423f1d93897b2a191fb6c9a15fb"
server_key_version: "kuduclusterkey@0"
I20260812 06:58:13.431406 14982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:13.444799 14982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:13.448805 14982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:13.449131 14982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:13.450105 14982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:13.451553 14982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:58:13.451664 14982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:13.451824 14982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:58:13.451901 14982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:13.487320 14982 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:44019
I20260812 06:58:13.487358 15208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:44019 every 8 connection(s)
I20260812 06:58:13.488678 14982 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:58:13.492245 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 14982
I20260812 06:58:13.492480 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:58:13.493105 27528 external_mini_cluster.cc:1428] Setting key ecd3efb1d57da637a4bfdfa257d6cebf
Aug 12 06:58:13 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:58:13.509302 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.491182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:45619 (local address 127.26.226.62:46465)
0812 06:58:13.491681 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:13.491688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:13.492951 (+ 1263us) server_negotiation.cc:407] Connection header received
0812 06:58:13.495239 (+ 2288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:13.495245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:13.495375 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:13.495493 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:13.497022 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.497753 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.499248 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.499691 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.499785 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.503058 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:13.503064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:13.503068 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:13.503102 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:13.506199 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:13.506945 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:13.506950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:13.506952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:13.507087 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:13.507541 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:13.507551 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:13.507553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:13.507818 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:13.507991 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:13.508684 (+ 693us) server_negotiation.cc:299] Negotiation successful
0812 06:58:13.508923 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":208,"threads_started":1}
I20260812 06:58:13.510242 15212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.491863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:46465 (local address 127.26.226.3:45619)
0812 06:58:13.492779 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:13.492826 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:13.494912 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:13.495696 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:13.495704 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:13.496278 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:13.496863 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.496881 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.497892 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.497896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.499105 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.499124 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.499873 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.499877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.500143 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.500778 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:13.500800 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:13.502806 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:13.506335 (+ 3529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:13.506341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:13.506353 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:13.506791 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:13.507202 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:13.507208 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:13.507213 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:13.507417 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:13.507952 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:13.507960 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:13.508336 (+ 376us) client_negotiation.cc:769] Sending connection context
0812 06:58:13.508588 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 06:58:13.509092 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":391,"threads_started":1}
I20260812 06:58:13.511520 15209 heartbeater.cc:347] Connected to a master server at 127.26.226.62:46465
I20260812 06:58:13.511847 15209 heartbeater.cc:464] Registering TS with master...
I20260812 06:58:13.512653 15209 heartbeater.cc:511] Master 127.26.226.62:46465 requested a full tablet report, sending...
I20260812 06:58:13.514174 14336 ts_manager.cc:194] Registered new tserver with Master: 7593b83a15d14273a0c5555b628d8dac (127.26.226.3:44019)
I20260812 06:58:13.514992 14336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:45619
I20260812 06:58:13.520962 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:58:13.538631 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.525738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45956 (local address 127.26.226.62:46465)
0812 06:58:13.526073 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:13.526081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:13.526378 (+ 297us) server_negotiation.cc:407] Connection header received
0812 06:58:13.526530 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:13.526534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:13.526609 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:13.526702 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:13.528198 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.528895 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.530368 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.530817 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.530932 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.533432 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:13.533437 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:13.533440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:13.533472 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:13.536309 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:13.536959 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:13.536963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:13.536965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:13.537095 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:13.537420 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:13.537424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:13.537426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:13.537669 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:13.537867 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:13.538126 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 06:58:13.538292 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260812 06:58:13.542727 14336 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:58:13.549813 14336 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:58:13.587891 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.562668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55756 (local address 127.26.226.1:40909)
0812 06:58:13.577164 (+ 14496us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:13.577173 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:13.577725 (+ 552us) server_negotiation.cc:407] Connection header received
0812 06:58:13.578216 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:13.578224 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:13.578490 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:13.578680 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:13.580295 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.581487 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.584808 (+ 3321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.586300 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.586417 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.586667 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:13.587340 (+ 673us) server_negotiation.cc:299] Negotiation successful
0812 06:58:13.587529 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14277,"thread_start_us":14503,"threads_started":1}
I20260812 06:58:13.588246 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.562827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:44019 (local address 127.0.0.1:38782)
0812 06:58:13.577777 (+ 14950us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:13.577794 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:13.577913 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:13.579004 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:13.579012 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:13.579056 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:13.580368 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.580383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.581751 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.581761 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.584231 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.584244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.587248 (+ 3004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.587257 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.587430 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.587464 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:58:13.587546 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 06:58:13.587775 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14823,"mutex_wait_us":110,"spinlock_wait_cycles":1280,"thread_start_us":14166,"threads_started":1}
I20260812 06:58:13.588392 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.562972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38782 (local address 127.26.226.3:44019)
0812 06:58:13.576861 (+ 13889us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:13.576872 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:13.577981 (+ 1109us) server_negotiation.cc:407] Connection header received
0812 06:58:13.578233 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:13.578241 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:13.578500 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:13.578680 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:13.580579 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.581596 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.584380 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.585849 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.586314 (+ 465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.587106 (+ 792us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:13.587641 (+ 535us) server_negotiation.cc:299] Negotiation successful
0812 06:58:13.587816 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13674,"thread_start_us":13485,"threads_started":1}
I20260812 06:58:13.588815 15233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.562492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:40909 (local address 127.0.0.1:55756)
0812 06:58:13.577484 (+ 14992us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:13.577551 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:13.577914 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:13.579004 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:13.579012 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:13.579063 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:13.579659 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.579678 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.581639 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.581647 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.584614 (+ 2967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.584629 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.586819 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.586826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.587030 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.587099 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 06:58:13.588230 (+ 1131us) client_negotiation.cc:240] Negotiation successful
0812 06:58:13.588393 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14681,"mutex_wait_us":261,"spinlock_wait_cycles":3456,"thread_start_us":14999,"threads_started":1}
I20260812 06:58:13.589505 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.578529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:38203 (local address 127.0.0.1:44244)
0812 06:58:13.580220 (+ 1691us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:13.580240 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:13.580396 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:13.582343 (+ 1947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:13.582349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:13.582379 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:13.582918 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.582932 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.584882 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.584892 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.586439 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.586454 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.588782 (+ 2328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.588787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.588959 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.588995 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:58:13.589092 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 06:58:13.589218 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1481,"thread_start_us":1315,"threads_started":1}
I20260812 06:58:13.590953 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.578888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.26.226.2:38203)
0812 06:58:13.579649 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:13.579658 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:13.580373 (+ 715us) server_negotiation.cc:407] Connection header received
0812 06:58:13.581785 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:13.581794 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:13.582062 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:13.582279 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:13.583149 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.584751 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.586631 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.587949 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.588362 (+ 413us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.588594 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:13.589327 (+ 733us) server_negotiation.cc:299] Negotiation successful
0812 06:58:13.589499 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":431,"threads_started":1}
I20260812 06:58:13.592633 15097 tablet_service.cc:1467] Processing CreateTablet for tablet 5c4296d4e8724504a90fd5c63b9e52d2 (DEFAULT_TABLE table=test-table [id=7c641bb539c044c5a783d1f517f844ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:58:13.592658 14590 tablet_service.cc:1467] Processing CreateTablet for tablet 5c4296d4e8724504a90fd5c63b9e52d2 (DEFAULT_TABLE table=test-table [id=7c641bb539c044c5a783d1f517f844ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:58:13.594063 15097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c4296d4e8724504a90fd5c63b9e52d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:13.594218 14590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c4296d4e8724504a90fd5c63b9e52d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:13.594889 14846 tablet_service.cc:1467] Processing CreateTablet for tablet 5c4296d4e8724504a90fd5c63b9e52d2 (DEFAULT_TABLE table=test-table [id=7c641bb539c044c5a783d1f517f844ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:58:13.596750 14846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c4296d4e8724504a90fd5c63b9e52d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:13.605839 15240 tablet_bootstrap.cc:492] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: Bootstrap starting.
I20260812 06:58:13.609272 15240 tablet_bootstrap.cc:654] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: Neither blocks nor log segments found. Creating new log.
I20260812 06:58:13.610396 15241 tablet_bootstrap.cc:492] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: Bootstrap starting.
I20260812 06:58:13.610740 15240 log.cc:824] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:13.613322 15243 tablet_bootstrap.cc:492] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: Bootstrap starting.
I20260812 06:58:13.614493 15240 tablet_bootstrap.cc:492] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: No bootstrap required, opened a new log
I20260812 06:58:13.614814 15240 ts_tablet_manager.cc:1397] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 06:58:13.615692 15241 tablet_bootstrap.cc:654] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: Neither blocks nor log segments found. Creating new log.
I20260812 06:58:13.617017 15243 tablet_bootstrap.cc:654] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: Neither blocks nor log segments found. Creating new log.
I20260812 06:58:13.617044 15241 log.cc:824] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:13.618280 15243 log.cc:824] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:13.621379 15241 tablet_bootstrap.cc:492] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: No bootstrap required, opened a new log
I20260812 06:58:13.621675 15241 ts_tablet_manager.cc:1397] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.006s
I20260812 06:58:13.622448 15243 tablet_bootstrap.cc:492] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: No bootstrap required, opened a new log
I20260812 06:58:13.622718 15243 ts_tablet_manager.cc:1397] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 06:58:13.624696 15240 raft_consensus.cc:348] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 } } peers { permanent_uuid: "7593b83a15d14273a0c5555b628d8dac" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44019 } } peers { permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 } }
I20260812 06:58:13.625082 15240 raft_consensus.cc:374] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:13.625188 15240 raft_consensus.cc:729] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7593b83a15d14273a0c5555b628d8dac, State: Initialized, Role: FOLLOWER
I20260812 06:58:13.625808 15240 consensus_queue.cc:260] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 } } peers { permanent_uuid: "7593b83a15d14273a0c5555b628d8dac" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44019 } } peers { permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 } }
I20260812 06:58:13.626933 15209 heartbeater.cc:503] Master 127.26.226.62:46465 was elected leader, sending a full tablet report...
I20260812 06:58:13.627596 15240 ts_tablet_manager.cc:1428] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260812 06:58:13.633215 15241 raft_consensus.cc:348] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 } } peers { permanent_uuid: "7593b83a15d14273a0c5555b628d8dac" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44019 } } peers { permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 } }
I20260812 06:58:13.633347 15243 raft_consensus.cc:348] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 } } peers { permanent_uuid: "7593b83a15d14273a0c5555b628d8dac" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44019 } } peers { permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 } }
I20260812 06:58:13.633751 15241 raft_consensus.cc:374] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:13.633751 15243 raft_consensus.cc:374] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:13.633863 15243 raft_consensus.cc:729] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 804cece7ba4b45dea8ff385aad7e9a8a, State: Initialized, Role: FOLLOWER
I20260812 06:58:13.633863 15241 raft_consensus.cc:729] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91bd6d5951354767aea850a5a80f4293, State: Initialized, Role: FOLLOWER
I20260812 06:58:13.634513 15241 consensus_queue.cc:260] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 } } peers { permanent_uuid: "7593b83a15d14273a0c5555b628d8dac" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44019 } } peers { permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 } }
I20260812 06:58:13.634505 15243 consensus_queue.cc:260] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 } } peers { permanent_uuid: "7593b83a15d14273a0c5555b628d8dac" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44019 } } peers { permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 } }
I20260812 06:58:13.636288 14971 heartbeater.cc:503] Master 127.26.226.62:46465 was elected leader, sending a full tablet report...
I20260812 06:58:13.636597 15241 ts_tablet_manager.cc:1428] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260812 06:58:13.636606 15243 ts_tablet_manager.cc:1428] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
W20260812 06:58:13.651002 14712 tablet.cc:2369] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:58:13.725683 15249 raft_consensus.cc:482] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:58:13.726037 15249 raft_consensus.cc:504] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 } } peers { permanent_uuid: "7593b83a15d14273a0c5555b628d8dac" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44019 } } peers { permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 } }
I20260812 06:58:13.727867 15249 leader_election.cc:288] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91bd6d5951354767aea850a5a80f4293 (127.26.226.1:40909), 804cece7ba4b45dea8ff385aad7e9a8a (127.26.226.2:38203)
I20260812 06:58:13.734338 15212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.728437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:38203 (local address 127.26.226.3:59639)
0812 06:58:13.728792 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:13.728821 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:13.728992 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:13.729434 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:13.729440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:13.729474 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:13.729698 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.729707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.730786 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.730792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.732209 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.732226 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.733529 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.733535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.733743 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.733783 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 06:58:13.733886 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 06:58:13.734035 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260812 06:58:13.734357 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.728440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:59639 (local address 127.26.226.2:38203)
0812 06:58:13.728669 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:13.728678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:13.728937 (+ 259us) server_negotiation.cc:407] Connection header received
0812 06:58:13.729204 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:13.729208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:13.729275 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:13.729412 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:13.729867 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.730620 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.732383 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.733379 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.733544 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.733619 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:13.733940 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 06:58:13.734099 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 06:58:13.735785 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.728632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:40785 (local address 127.26.226.1:40909)
0812 06:58:13.728838 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:13.728842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:13.729909 (+ 1067us) server_negotiation.cc:407] Connection header received
0812 06:58:13.730110 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:13.730115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:13.730182 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:13.730291 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:13.730856 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.731568 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.733460 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.734803 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.734994 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.735069 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:13.735378 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 06:58:13.735531 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:58:13.735605 14881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c4296d4e8724504a90fd5c63b9e52d2" candidate_uuid: "7593b83a15d14273a0c5555b628d8dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" is_pre_election: true
I20260812 06:58:13.735870 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.728952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:40909 (local address 127.26.226.3:40785)
0812 06:58:13.729797 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:13.729811 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:13.729957 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:13.730310 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:13.730314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:13.730337 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:13.730684 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.730692 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.731746 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.731755 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.733285 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:13.733303 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.735018 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.735025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:13.735203 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.735228 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 06:58:13.735337 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 06:58:13.735477 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":459,"threads_started":1}
I20260812 06:58:13.736224 14881 raft_consensus.cc:2454] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7593b83a15d14273a0c5555b628d8dac in term 0.
I20260812 06:58:13.737059 15017 leader_election.cc:302] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7593b83a15d14273a0c5555b628d8dac, 804cece7ba4b45dea8ff385aad7e9a8a; no voters:
I20260812 06:58:13.737138 14624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c4296d4e8724504a90fd5c63b9e52d2" candidate_uuid: "7593b83a15d14273a0c5555b628d8dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91bd6d5951354767aea850a5a80f4293" is_pre_election: true
I20260812 06:58:13.737564 15249 raft_consensus.cc:2781] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:58:13.737740 15249 raft_consensus.cc:482] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:58:13.737725 14624 raft_consensus.cc:2454] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7593b83a15d14273a0c5555b628d8dac in term 0.
I20260812 06:58:13.737865 15249 raft_consensus.cc:3037] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 0 FOLLOWER]: Advancing to term 1
W20260812 06:58:13.740389 15210 tablet.cc:2369] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:58:13.740537 15249 raft_consensus.cc:504] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 } } peers { permanent_uuid: "7593b83a15d14273a0c5555b628d8dac" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44019 } } peers { permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 } }
I20260812 06:58:13.741377 15249 leader_election.cc:288] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [CANDIDATE]: Term 1 election: Requested vote from peers 91bd6d5951354767aea850a5a80f4293 (127.26.226.1:40909), 804cece7ba4b45dea8ff385aad7e9a8a (127.26.226.2:38203)
I20260812 06:58:13.741868 14624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c4296d4e8724504a90fd5c63b9e52d2" candidate_uuid: "7593b83a15d14273a0c5555b628d8dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91bd6d5951354767aea850a5a80f4293"
I20260812 06:58:13.741930 14881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c4296d4e8724504a90fd5c63b9e52d2" candidate_uuid: "7593b83a15d14273a0c5555b628d8dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "804cece7ba4b45dea8ff385aad7e9a8a"
I20260812 06:58:13.742097 14624 raft_consensus.cc:3037] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:58:13.742159 14881 raft_consensus.cc:3037] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:58:13.744539 14624 raft_consensus.cc:2454] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7593b83a15d14273a0c5555b628d8dac in term 1.
I20260812 06:58:13.744705 14881 raft_consensus.cc:2454] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7593b83a15d14273a0c5555b628d8dac in term 1.
I20260812 06:58:13.745091 15014 leader_election.cc:302] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7593b83a15d14273a0c5555b628d8dac, 91bd6d5951354767aea850a5a80f4293; no voters:
I20260812 06:58:13.745447 15249 raft_consensus.cc:2781] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:58:13.746120 15249 raft_consensus.cc:686] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 1 LEADER]: Becoming Leader. State: Replica: 7593b83a15d14273a0c5555b628d8dac, State: Running, Role: LEADER
I20260812 06:58:13.746645 15249 consensus_queue.cc:237] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 } } peers { permanent_uuid: "7593b83a15d14273a0c5555b628d8dac" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44019 } } peers { permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 } }
I20260812 06:58:13.752159 14336 catalog_manager.cc:5495] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac reported cstate change: term changed from 0 to 1, leader changed from <none> to 7593b83a15d14273a0c5555b628d8dac (127.26.226.3). New cstate: current_term: 1 leader_uuid: "7593b83a15d14273a0c5555b628d8dac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7593b83a15d14273a0c5555b628d8dac" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 } health_report { overall_health: UNKNOWN } } }
I20260812 06:58:13.784816 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.778982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.26.226.3:44019)
0812 06:58:13.779192 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:13.779200 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:13.779291 (+ 91us) server_negotiation.cc:407] Connection header received
0812 06:58:13.779562 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:13.779566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:13.779654 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:13.779818 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:58:13.780267 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.781098 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.782566 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.782999 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.783137 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.783663 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:58:13.784223 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:58:13.784393 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:13.784456 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 06:58:13.784537 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:58:13.793269 14624 raft_consensus.cc:1261] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293 [term 1 FOLLOWER]: Refusing update from remote peer 7593b83a15d14273a0c5555b628d8dac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:58:13.793277 14881 raft_consensus.cc:1261] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a [term 1 FOLLOWER]: Refusing update from remote peer 7593b83a15d14273a0c5555b628d8dac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:58:13.794265 15255 consensus_queue.cc:1035] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [LEADER]: Connected to new peer: Peer: permanent_uuid: "91bd6d5951354767aea850a5a80f4293" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:58:13.794679 15249 consensus_queue.cc:1035] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [LEADER]: Connected to new peer: Peer: permanent_uuid: "804cece7ba4b45dea8ff385aad7e9a8a" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:58:13.806025 15260 mvcc.cc:205] Tried to move back new op lower bound from 7317577292966576128 to 7317577292790915072. Current Snapshot: MvccSnapshot[applied={T|T < 7317577292966576128}]
I20260812 06:58:13.822151 14333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:58:13.822556 14333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:58:13.826877 14333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed66c844a6f34194a16ba4bb8c24988f: Sending DeleteTablet for 3 replicas of tablet 5c4296d4e8724504a90fd5c63b9e52d2
I20260812 06:58:13.828313 14590 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c4296d4e8724504a90fd5c63b9e52d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:58:13 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:55756
I20260812 06:58:13.828725 14846 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c4296d4e8724504a90fd5c63b9e52d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:58:13 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:44244
I20260812 06:58:13.829035 15097 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c4296d4e8724504a90fd5c63b9e52d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:58:13 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:38782
I20260812 06:58:13.831396 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 14471
I20260812 06:58:13.842129 15272 tablet_replica.cc:330] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: stopping tablet replica
I20260812 06:58:13.842238 15273 tablet_replica.cc:330] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: stopping tablet replica
I20260812 06:58:13.842938 15271 tablet_replica.cc:330] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: stopping tablet replica
I20260812 06:58:13.842887 15273 raft_consensus.cc:2229] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:58:13.843443 15271 raft_consensus.cc:2229] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:58:13.843667 15273 raft_consensus.cc:2258] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:58:13.843992 15272 raft_consensus.cc:2229] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:58:13.844632 15272 raft_consensus.cc:2258] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:58:13.845166 15271 raft_consensus.cc:2258] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:58:13.846575 15273 ts_tablet_manager.cc:1905] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:58:13.848249 15271 ts_tablet_manager.cc:1905] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:58:13.849660 15272 ts_tablet_manager.cc:1905] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 06:58:13 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:58:13.856036 15271 ts_tablet_manager.cc:1918] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:58:13.856276 15271 log.cc:1196] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/5c4296d4e8724504a90fd5c63b9e52d2
I20260812 06:58:13.856772 15272 ts_tablet_manager.cc:1918] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:58:13.856940 15271 ts_tablet_manager.cc:1939] T 5c4296d4e8724504a90fd5c63b9e52d2 P 91bd6d5951354767aea850a5a80f4293: Deleting consensus metadata
I20260812 06:58:13.859091 14311 catalog_manager.cc:4841] TS 91bd6d5951354767aea850a5a80f4293 (127.26.226.1:40909): tablet 5c4296d4e8724504a90fd5c63b9e52d2 (table test-table [id=7c641bb539c044c5a783d1f517f844ae]) successfully deleted
I20260812 06:58:13.859530 15272 log.cc:1196] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/5c4296d4e8724504a90fd5c63b9e52d2
I20260812 06:58:13.860227 15272 ts_tablet_manager.cc:1939] T 5c4296d4e8724504a90fd5c63b9e52d2 P 804cece7ba4b45dea8ff385aad7e9a8a: Deleting consensus metadata
I20260812 06:58:13.862264 14313 catalog_manager.cc:4841] TS 804cece7ba4b45dea8ff385aad7e9a8a (127.26.226.2:38203): tablet 5c4296d4e8724504a90fd5c63b9e52d2 (table test-table [id=7c641bb539c044c5a783d1f517f844ae]) successfully deleted
I20260812 06:58:13.864318 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.832721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.26.226.1:40909)
0812 06:58:13.832928 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:13.832935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:13.837369 (+ 4434us) server_negotiation.cc:407] Connection header received
0812 06:58:13.837620 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:13.837625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:13.837755 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:13.837878 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:13.844957 (+ 7079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.846218 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.848416 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.849125 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.849280 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:13.855149 (+ 5869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:13.855167 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:13.855189 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:13.855240 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:13.859879 (+ 4639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:13.860999 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:13.861012 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:13.861016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:13.861202 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:13.861720 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:13.861726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:13.861730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:13.862696 (+ 966us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:13.863031 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:13.863379 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 06:58:13.863666 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":77}
I20260812 06:58:13.864602 15273 ts_tablet_manager.cc:1918] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:58:13.864818 15273 log.cc:1196] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/5c4296d4e8724504a90fd5c63b9e52d2
I20260812 06:58:13.865082 14698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:58:13.865507 15273 ts_tablet_manager.cc:1939] T 5c4296d4e8724504a90fd5c63b9e52d2 P 7593b83a15d14273a0c5555b628d8dac: Deleting consensus metadata
I20260812 06:58:13.868002 14313 catalog_manager.cc:4841] TS 7593b83a15d14273a0c5555b628d8dac (127.26.226.3:44019): tablet 5c4296d4e8724504a90fd5c63b9e52d2 (table test-table [id=7c641bb539c044c5a783d1f517f844ae]) successfully deleted
I20260812 06:58:13.960311 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 14471
I20260812 06:58:13.991698 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 14717
Aug 12 06:58:13 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:58:14.008625 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:13.992683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.26.226.2:38203)
0812 06:58:13.992866 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:13.992872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:13.992927 (+ 55us) server_negotiation.cc:407] Connection header received
0812 06:58:13.993210 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:13.993214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:13.993285 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:13.993399 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:13.994863 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.995630 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.997022 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:13.997419 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:13.997563 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:14.002270 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:14.002287 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:14.002304 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:14.002355 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:14.005414 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:14.006202 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:14.006210 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:14.006212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:14.006351 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:14.006762 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:14.006767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:14.006770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:14.007191 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:14.007508 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:14.007843 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 06:58:14.008230 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:58:14.009485 14960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:58:14.101009 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 14717
I20260812 06:58:14.137763 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 14982
Aug 12 06:58:14 dist-test-slave-t3q3 krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:58:14.156296 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:14.138707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.26.226.3:44019)
0812 06:58:14.138900 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:14.138908 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:14.139020 (+ 112us) server_negotiation.cc:407] Connection header received
0812 06:58:14.139271 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:14.139280 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:14.139359 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:14.139473 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:14.140832 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:14.141673 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:14.142974 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:14.143529 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:14.143688 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:14.148831 (+ 5143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:14.148847 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:14.148866 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:14.148918 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:14.153156 (+ 4238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:14.153902 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:14.153912 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:14.153916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:14.154120 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:14.154517 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:14.154528 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:14.154533 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:14.155043 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:14.155401 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:14.155580 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 06:58:14.155819 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:58:14.157091 15202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:58:14.281306 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 14982
I20260812 06:58:14.311900 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 14278
I20260812 06:58:14.313223 14386 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:58:14.429072 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 14278
I20260812 06:58:14.463925 27528 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 06:58:14.811022 27528 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 06:58:14.811295 27528 mini_ranger.cc:66] Stopping Ranger...
I20260812 06:58:15.158380 27528 mini_ranger.cc:68] Stopped Ranger
2026-08-12 06:58:15.158 UTC [13532] LOG: received smart shutdown request
2026-08-12 06:58:15.163 UTC [13532] LOG: background worker "logical replication launcher" (PID 13550) exited with exit code 1
2026-08-12 06:58:15.163 UTC [13539] LOG: shutting down
2026-08-12 06:58:15.348 UTC [13532] LOG: database system is shut down
2026-08-12T06:58:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52982 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:58:15 dist-test-slave-t3q3 krb5kdc[15294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:58:15 dist-test-slave-t3q3 krb5kdc[15294](info): set up 2 sockets
Aug 12 06:58:15 dist-test-slave-t3q3 krb5kdc[15294](info): commencing operation
krb5kdc: starting...
W20260812 06:58:16.917937 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.368s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:58:17 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:58:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:58:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.226.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.226.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/rangeradmin_127.26.226.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/rangeradmin_127.26.226.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.226.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.226.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/rangerlookup_127.26.226.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/rangerlookup_127.26.226.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.226.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.226.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/HTTP_127.26.226.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/HTTP_127.26.226.20@KRBTEST.COM.keytab.
I20260812 06:58:17.357429 27528 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6poUc4/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/postgres -l logfile start
2026-08-12 06:58:19.978 UTC [15352] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 06:58:19.978 UTC [15352] LOG: listening on IPv4 address "127.26.226.20", port 53019
2026-08-12 06:58:19.982 UTC [15352] LOG: listening on Unix socket "/tmp/.s.PGSQL.53019"
2026-08-12 06:58:20.006 UTC [15354] LOG: database system was shut down at 2026-08-12 06:58:18 UTC
2026-08-12 06:58:20.010 UTC [15352] LOG: database system is ready to accept connections
I20260812 06:58:21.243247 27528 mini_postgres.cc:96] Postgres bound to 53019
2026-08-12 06:58:21.268 UTC [15365] FATAL: database "slave" does not exist
127.26.226.20:53019 - accepting connections
I20260812 06:58:21.269510 27528 mini_ranger.cc:161] Starting Ranger...
I20260812 06:58:21.305146 27528 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 06:58:21.771790 27528 mini_ranger.cc:87] Created ranger Postgres database
I20260812 06:58:21.771937 27528 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-admin
2026-08-12 06:58:22,074 [I] DB FLAVOR :POSTGRES
2026-08-12 06:58:22,075 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:58:22,075 [I] Checking connection..
2026-08-12 06:58:22,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 06:58:22,321 [I] Checking connection passed.
2026-08-12 06:58:22,322 [I] --------- Verifying version history table ---------
2026-08-12 06:58:22,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:58:22,585 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:58:22,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:58:22,867 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:58:22,867 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 06:58:22,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 06:58:23.131 UTC [15474] WARNING: there is no transaction in progress
2026-08-12 06:58:23,142 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 06:58:23,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:58:23,397 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 06:58:23,398 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:58:23,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 06:58:23,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 06:58:23,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t3q3.c.gcp-upstream.internal','N') ;"
2026-08-12 06:58:24,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:58:24,450 [I] Table x_portal_user does not exist in database ranger
2026-08-12 06:58:24,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:58:24,719 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 06:58:24,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:58:24,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:58:25,235 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 06:58:25,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 06:58:26.240 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.256 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.272 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.287 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.306 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.327 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.383 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.390 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.404 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.414 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.424 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.438 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.447 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.453 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.460 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:26.466 UTC [15662] WARNING: there is no transaction in progress
2026-08-12 06:58:27,114 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 06:58:27,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 06:58:27,378 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 06:58:27,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:58:27,621 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 06:58:27,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:58:27,863 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 06:58:27,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:58:28,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:58:28,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t3q3.c.gcp-upstream.internal';"
2026-08-12 06:58:28,651 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 06:58:28,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 06:58:28,907 [I] DB_PATCHES have already been applied
I20260812 06:58:28.914295 27528 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-admin:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6poUc4/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 06:58:28.914386 27528 mini_ranger.cc:193] Using host: 127.26.226.20
I20260812 06:58:28.936295 27528 mini_ranger.cc:227] Ranger admin URL: http://127.26.226.20:38069
Aug 12, 2026 6:58:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:58:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:58:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 6:58:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 6:58:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 6:58:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 6:58:29 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 06:58:30.373052 27528 mini_ranger.cc:160] Time spent starting Ranger: real 9.104s user 0.003s sys 0.066s
Aug 12, 2026 6:58:51 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 6:58:51 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 6:58:51 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 6:58:51 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 6:58:52 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 06:58:53.171201 27528 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.226.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.226.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/rangerkms_127.26.226.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.226.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/rangerkms_127.26.226.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.226.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.226.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.226.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/HTTP_127.26.226.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.226.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/HTTP_127.26.226.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 06:58:53 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517933, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 06:58:53.418265 27528 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 06:58:53.448639 27528 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 06:58:54.378575 27528 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 06:58:54.378726 27528 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-kms
I20260812 06:58:54.378773 27528 mini_ranger_kms.cc:227] Using postgres at 127.26.226.20:53019
2026-08-12 06:58:54,793 [I] DB FLAVOR :POSTGRES
2026-08-12 06:58:54,793 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:58:54,793 [I] Checking connection
2026-08-12 06:58:54,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 06:58:55,040 [I] connection success
2026-08-12 06:58:55,041 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 06:58:55,041 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 06:58:55,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 06:58:55,311 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 06:58:55,312 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:58:55,312 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 06:58:55,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.226.20:53019/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 06:58:55,592 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 06:58:56.129719 27528 mini_ranger_kms.cc:310] Created kms service
I20260812 06:58:56.321588 27528 mini_ranger_kms.cc:326] Created kudu user
I20260812 06:58:56.379644 27528 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 06:58:56 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517933, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:58:56.804879 27528 mini_ranger_kms.cc:384] Added ranger policy
I20260812 06:58:56.805284 27528 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-kms:/tmp/dist-test-task6poUc4/build/asan/bin/postgresql.jar:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6poUc4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 06:58:56.805358 27528 mini_ranger_kms.cc:240] Using host: 127.26.226.20
I20260812 06:58:56.828701 27528 mini_ranger_kms.cc:276] Ranger KMS PID: 15996
I20260812 06:58:56.828850 27528 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.226.20:56507
Aug 12, 2026 6:58:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:58:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:58:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 6:58:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 6:58:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 6:58:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 06:58:59 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517939, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.226.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 06:58:59 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517939, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.226.20@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:58:59.751933 27528 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.334s user 0.090s sys 0.365s
I20260812 06:58:59.752242 27528 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 06:58:59.752429 27528 mini_ranger_kms.cc:401] 127.26.226.20:56507/kms/v1/keys
Aug 12 06:59:00 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517939, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.226.20@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
Aug 12 06:59:02 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:59:02.463279 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:39023
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:39023
--ranger_config_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:56507/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:02.832617 16083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:02.832984 16083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:02.833143 16083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:02.840888 16083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:02.841027 16083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:02.841114 16083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:02.841236 16083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:02.841322 16083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:02.852313 16083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.226.20:56507/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:39023
--ranger_config_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:39023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:02.853776 16083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:02.855189 16083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:02.869812 16092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:02.870046 16096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:02.870124 16093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:02.870759 16083 server_base.cc:1034] running on GCE node
I20260812 06:59:02.871666 16083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:02.873353 16083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:02.874594 16083 hybrid_clock.cc:648] HybridClock initialized: now 1786517942874545 us; error 44 us; skew 500 ppm
I20260812 06:59:02.874977 16083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:02 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517942, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:02.881314 16083 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:59:02.883131 16083 webserver.cc:460] Webserver started at http://127.26.226.62:43789/ using document root <none> and password file <none>
I20260812 06:59:02.883975 16083 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:02.884136 16083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:02.884497 16083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:59:02 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517942, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:59:02.935315 16083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "a502e373644146d1a5a51aa2db8ebc1d"
format_stamp: "Formatted at 2026-08-12 06:59:02 on dist-test-slave-t3q3"
server_key: "8cec1205295e5b4372df237efe1a4904"
server_key_iv: "66778fc1357193fb38e0b0f62c901824"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:02.936367 16083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "a502e373644146d1a5a51aa2db8ebc1d"
format_stamp: "Formatted at 2026-08-12 06:59:02 on dist-test-slave-t3q3"
server_key: "8cec1205295e5b4372df237efe1a4904"
server_key_iv: "66778fc1357193fb38e0b0f62c901824"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:02.942144 16083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:59:02.959338 16108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:02.960980 16083 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:59:02.961185 16083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "a502e373644146d1a5a51aa2db8ebc1d"
format_stamp: "Formatted at 2026-08-12 06:59:02 on dist-test-slave-t3q3"
server_key: "8cec1205295e5b4372df237efe1a4904"
server_key_iv: "66778fc1357193fb38e0b0f62c901824"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:02.961439 16083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:02.973675 16083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:02.977792 16083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:02.978188 16083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:02.998200 16083 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:39023
I20260812 06:59:02.998236 16198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:39023 every 8 connection(s)
I20260812 06:59:02.999648 16083 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:03.003520 16201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:03.009063 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 16083
I20260812 06:59:03.009326 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:59:03.010031 27528 external_mini_cluster.cc:1428] Setting key a6c6382f0374716958f50954d430632e
I20260812 06:59:03.014914 16201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: Bootstrap starting.
I20260812 06:59:03.018754 16201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:03.020093 16201 log.cc:824] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:03.023862 16201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: No bootstrap required, opened a new log
Aug 12 06:59:03 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:03.032363 16201 raft_consensus.cc:348] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } }
I20260812 06:59:03.032779 16201 raft_consensus.cc:374] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:03.032924 16201 raft_consensus.cc:729] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a502e373644146d1a5a51aa2db8ebc1d, State: Initialized, Role: FOLLOWER
I20260812 06:59:03.033629 16201 consensus_queue.cc:260] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } }
I20260812 06:59:03.033901 16201 raft_consensus.cc:388] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:03.034065 16201 raft_consensus.cc:482] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:03.034266 16201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:03.034662 16206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:03.012104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43112 (local address 127.26.226.62:39023)
0812 06:59:03.013040 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:03.013052 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:03.013093 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:59:03.014771 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:03.014793 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:03.015218 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:03.015532 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:03.017326 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:03.018578 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:03.020504 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:03.021670 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:03.021818 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:03.026422 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:03.026434 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:03.026455 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:03.026505 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:03.029957 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:03.030827 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:03.030835 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:03.030842 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:03.031015 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:03.031432 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:03.031436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:03.031438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:03.032039 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:03.032425 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:03.032891 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 06:59:03.033420 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":433,"threads_started":1}
I20260812 06:59:03.036885 16201 raft_consensus.cc:504] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } }
I20260812 06:59:03.037444 16201 leader_election.cc:302] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a502e373644146d1a5a51aa2db8ebc1d; no voters:
I20260812 06:59:03.038151 16201 leader_election.cc:288] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:03.038295 16211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:03.039038 16211 raft_consensus.cc:686] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 1 LEADER]: Becoming Leader. State: Replica: a502e373644146d1a5a51aa2db8ebc1d, State: Running, Role: LEADER
I20260812 06:59:03.039672 16211 consensus_queue.cc:237] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } }
I20260812 06:59:03.040541 16201 sys_catalog.cc:564] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:03.042985 16213 sys_catalog.cc:455] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a502e373644146d1a5a51aa2db8ebc1d. Latest consensus state: current_term: 1 leader_uuid: "a502e373644146d1a5a51aa2db8ebc1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } } }
I20260812 06:59:03.043455 16213 sys_catalog.cc:458] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:03.042987 16212 sys_catalog.cc:455] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a502e373644146d1a5a51aa2db8ebc1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } } }
I20260812 06:59:03.043859 16212 sys_catalog.cc:458] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:03.045251 16216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:03.050334 16216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:03.059358 16216 catalog_manager.cc:1347] Generated new cluster ID: 74d519394b6940edbcb1baeb3d38e24c
I20260812 06:59:03.059582 16216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:03.064258 16201 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 06:59:03.091845 16216 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:03.094146 16216 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:03.108181 16216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: Generated new TSK 0
I20260812 06:59:03.109455 16216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:59:03.924827 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:56507/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39023
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:59:04 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517944, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 06:59:04.381665 16271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:04.382201 16271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:04.382436 16271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:04.394313 16271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:04.394562 16271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:04.394796 16271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:59:04.410943 16271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.226.20:56507/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:04.413012 16271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:04.414813 16271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:04.433063 16285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:04.434027 16287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:04.442679 16290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:04.443049 16271 server_base.cc:1034] running on GCE node
I20260812 06:59:04.443934 16271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:04.445349 16271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:04.446703 16271 hybrid_clock.cc:648] HybridClock initialized: now 1786517944446577 us; error 112 us; skew 500 ppm
I20260812 06:59:04.447119 16271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:04 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517944, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:04.451774 16271 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:59:04.455892 16271 webserver.cc:460] Webserver started at http://127.26.226.1:39233/ using document root <none> and password file <none>
I20260812 06:59:04.457115 16271 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:04.457374 16271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:04.457851 16271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:59:04 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517944, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:59:04.479198 16271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "78d93c67bd7049139125897dfbea521a"
format_stamp: "Formatted at 2026-08-12 06:59:04 on dist-test-slave-t3q3"
server_key: "fb251ab1c8f09b6f8723b9da55e30268"
server_key_iv: "2c4f9ae2e34cbced38ca78c0ee20b6dc"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:04.480612 16271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "78d93c67bd7049139125897dfbea521a"
format_stamp: "Formatted at 2026-08-12 06:59:04 on dist-test-slave-t3q3"
server_key: "fb251ab1c8f09b6f8723b9da55e30268"
server_key_iv: "2c4f9ae2e34cbced38ca78c0ee20b6dc"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:04.496052 16271 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.004s sys 0.009s
I20260812 06:59:04.513337 16300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:04.514973 16271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:59:04.515192 16271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "78d93c67bd7049139125897dfbea521a"
format_stamp: "Formatted at 2026-08-12 06:59:04 on dist-test-slave-t3q3"
server_key: "fb251ab1c8f09b6f8723b9da55e30268"
server_key_iv: "2c4f9ae2e34cbced38ca78c0ee20b6dc"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:04.515444 16271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:04.555290 16271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:04.561311 16271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:04.561769 16271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:04.565148 16271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:04.567204 16271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:04.567306 16271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:04.567426 16271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:04.567493 16271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:04.623582 16271 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:39065
I20260812 06:59:04.623817 16487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:39065 every 8 connection(s)
I20260812 06:59:04.625229 16271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:04.632881 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 16271
I20260812 06:59:04.633136 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:59:04.633916 27528 external_mini_cluster.cc:1428] Setting key d10f309be2dab145ad0993f07fc92842
Aug 12 06:59:04 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517944, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:04.668630 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:04.628664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:36159 (local address 127.26.226.62:39023)
0812 06:59:04.644410 (+ 15746us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:04.644417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:04.644444 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:59:04.645755 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:04.645760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:04.645835 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:04.645928 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:04.648240 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:04.649377 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:04.651906 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:04.652657 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:04.652769 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:04.658560 (+ 5791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:04.658568 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:04.658573 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:04.658621 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:04.662381 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:04.665732 (+ 3351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:04.665738 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:04.665745 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:04.665917 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:04.666499 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:04.666504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:04.666507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:04.666854 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:04.666983 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:04.667986 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0812 06:59:04.668268 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15611,"thread_start_us":15483,"threads_started":1}
I20260812 06:59:04.671088 16491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:04.629396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:39023 (local address 127.26.226.1:36159)
0812 06:59:04.642210 (+ 12814us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:04.642288 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:04.645311 (+ 3023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:04.646394 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:04.646408 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:04.647097 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:04.647982 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:04.648023 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:04.649596 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:04.649601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:04.651717 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:04.651742 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:04.653168 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:04.653177 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:04.653483 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:04.654424 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:04.654456 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:04.657474 (+ 3018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:04.664879 (+ 7405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:04.664890 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:04.664911 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:04.665570 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:04.666087 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:04.666091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:04.666095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:04.666366 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:04.667090 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:04.667098 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:04.667544 (+ 446us) client_negotiation.cc:769] Sending connection context
0812 06:59:04.669184 (+ 1640us) client_negotiation.cc:240] Negotiation successful
0812 06:59:04.669744 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12389,"thread_start_us":14866,"threads_started":1}
I20260812 06:59:04.673220 16488 heartbeater.cc:347] Connected to a master server at 127.26.226.62:39023
I20260812 06:59:04.673534 16488 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:04.674405 16488 heartbeater.cc:511] Master 127.26.226.62:39023 requested a full tablet report, sending...
I20260812 06:59:04.676986 16137 ts_manager.cc:194] Registered new tserver with Master: 78d93c67bd7049139125897dfbea521a (127.26.226.1:39065)
I20260812 06:59:04.678982 16137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:36159
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Aug 12 06:59:04 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517944, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:59:04.744033 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:56507/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39023
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:59:05 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517944, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
W20260812 06:59:05.261906 16496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:05.262427 16496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:05.262617 16496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:05.274322 16496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:05.274479 16496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:05.274719 16496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:59:05.290659 16496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.226.20:56507/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:05.292764 16496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:05.294718 16496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:05.313827 16512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:05.313848 16513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:05.318897 16496 server_base.cc:1034] running on GCE node
W20260812 06:59:05.318778 16518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:05.319945 16496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:05.321136 16496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:05.322386 16496 hybrid_clock.cc:648] HybridClock initialized: now 1786517945322308 us; error 72 us; skew 500 ppm
I20260812 06:59:05.322764 16496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:05 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:05.327075 16496 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:59:05.329193 16496 webserver.cc:460] Webserver started at http://127.26.226.2:34089/ using document root <none> and password file <none>
I20260812 06:59:05.330169 16496 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:05.330315 16496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:05.330823 16496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:59:05 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:59:05.348948 16496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "88c9932baa8f4b2bb823c1e2912d374f"
format_stamp: "Formatted at 2026-08-12 06:59:05 on dist-test-slave-t3q3"
server_key: "3b77e77a5d2598fd5610c5354b98e04f"
server_key_iv: "f56c1bdad17212ba21b0e329a5aff08a"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:05.349889 16496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "88c9932baa8f4b2bb823c1e2912d374f"
format_stamp: "Formatted at 2026-08-12 06:59:05 on dist-test-slave-t3q3"
server_key: "3b77e77a5d2598fd5610c5354b98e04f"
server_key_iv: "f56c1bdad17212ba21b0e329a5aff08a"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:05.356256 16496 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260812 06:59:05.376410 16534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:05.378398 16496 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:59:05.378604 16496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "88c9932baa8f4b2bb823c1e2912d374f"
format_stamp: "Formatted at 2026-08-12 06:59:05 on dist-test-slave-t3q3"
server_key: "3b77e77a5d2598fd5610c5354b98e04f"
server_key_iv: "f56c1bdad17212ba21b0e329a5aff08a"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:05.378868 16496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:05.398674 16496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:05.403977 16496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:05.404342 16496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:05.405555 16496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:05.407400 16496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:05.407500 16496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:05.407683 16496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:05.407771 16496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:05.451690 16496 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:37821
I20260812 06:59:05.451742 16707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:37821 every 8 connection(s)
I20260812 06:59:05.453564 16496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:05.454475 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 16496
I20260812 06:59:05.454651 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:59:05.455401 27528 external_mini_cluster.cc:1428] Setting key 115dcd50770fb2d77c3aef1f61b2ca65
Aug 12 06:59:05 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
I20260812 06:59:05.512387 16716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:05.470521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:39023 (local address 127.26.226.2:46999)
0812 06:59:05.485104 (+ 14583us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:05.485214 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:05.488719 (+ 3505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:05.490059 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:05.490075 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:05.490821 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:05.491744 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:05.491775 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:05.493548 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:05.493556 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:05.495880 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:05.495917 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:05.497440 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:05.497450 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:05.497816 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:05.498781 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:05.498817 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:05.501803 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:05.506611 (+ 4808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:05.506624 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:05.506645 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:05.507711 (+ 1066us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:05.508375 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:05.508380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:05.508384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:05.508585 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:05.509318 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:05.509326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:05.509816 (+ 490us) client_negotiation.cc:769] Sending connection context
0812 06:59:05.510157 (+ 341us) client_negotiation.cc:240] Negotiation successful
0812 06:59:05.510762 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12951,"thread_start_us":13251,"threads_started":1}
I20260812 06:59:05.513993 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:05.469806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:46999 (local address 127.26.226.62:39023)
0812 06:59:05.485081 (+ 15275us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:05.485088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:05.489220 (+ 4132us) server_negotiation.cc:407] Connection header received
0812 06:59:05.489306 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:05.489313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:05.489432 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:05.489540 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:05.491998 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:05.493326 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:05.496161 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:05.496973 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:05.497108 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:05.502127 (+ 5019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:05.502135 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:05.502142 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:05.502193 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:05.506390 (+ 4197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:05.508053 (+ 1663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:05.508084 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:05.508088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:05.508234 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:05.508723 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:05.508729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:05.508732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:05.509169 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:05.509328 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:05.513228 (+ 3900us) server_negotiation.cc:299] Negotiation successful
0812 06:59:05.513507 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15130,"thread_start_us":15274,"threads_started":1}
I20260812 06:59:05.515453 16710 heartbeater.cc:347] Connected to a master server at 127.26.226.62:39023
I20260812 06:59:05.515803 16710 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:05.516789 16710 heartbeater.cc:511] Master 127.26.226.62:39023 requested a full tablet report, sending...
I20260812 06:59:05.518577 16137 ts_manager.cc:194] Registered new tserver with Master: 88c9932baa8f4b2bb823c1e2912d374f (127.26.226.2:37821)
I20260812 06:59:05.519480 16137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:46999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:59:05.551853 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:56507/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39023
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:05.683305 16488 heartbeater.cc:503] Master 127.26.226.62:39023 was elected leader, sending a full tablet report...
W20260812 06:59:05.917872 16720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:05.918211 16720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:05.918417 16720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:05.925940 16720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:05.926105 16720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:05.926285 16720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:59:05.936630 16720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.226.20:56507/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:39023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:05.937990 16720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:05.939352 16720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:05.953085 16728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:05.953166 16730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:05.953198 16727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:05.954282 16720 server_base.cc:1034] running on GCE node
I20260812 06:59:05.954931 16720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:05.955946 16720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:05.957185 16720 hybrid_clock.cc:648] HybridClock initialized: now 1786517945957139 us; error 83 us; skew 500 ppm
I20260812 06:59:05.957513 16720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:05 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:05.960810 16720 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:59:05.962431 16720 webserver.cc:460] Webserver started at http://127.26.226.3:42435/ using document root <none> and password file <none>
I20260812 06:59:05.963160 16720 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:05.963315 16720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:05.963646 16720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:59:05 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:59:05.981312 16720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "d548df9f89be4ee788645e1e43d2d574"
format_stamp: "Formatted at 2026-08-12 06:59:05 on dist-test-slave-t3q3"
server_key: "7582820385834d4d307add2e6ffe5984"
server_key_iv: "b3b64aab762634865e37996ce5dfadfa"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:05.982474 16720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "d548df9f89be4ee788645e1e43d2d574"
format_stamp: "Formatted at 2026-08-12 06:59:05 on dist-test-slave-t3q3"
server_key: "7582820385834d4d307add2e6ffe5984"
server_key_iv: "b3b64aab762634865e37996ce5dfadfa"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:05.989646 16720 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260812 06:59:06.010079 16739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:06.011987 16720 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 06:59:06.012219 16720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "d548df9f89be4ee788645e1e43d2d574"
format_stamp: "Formatted at 2026-08-12 06:59:05 on dist-test-slave-t3q3"
server_key: "7582820385834d4d307add2e6ffe5984"
server_key_iv: "b3b64aab762634865e37996ce5dfadfa"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:06.012498 16720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:06.032959 16720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:06.036892 16720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:06.037159 16720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:06.038089 16720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:06.039430 16720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:06.039506 16720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:06.039593 16720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:06.039645 16720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:06.075827 16720 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:39695
I20260812 06:59:06.075878 16919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:39695 every 8 connection(s)
I20260812 06:59:06.077275 16720 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:06.086584 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 16720
I20260812 06:59:06.086807 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:59:06.087271 27528 external_mini_cluster.cc:1428] Setting key 5fa8a829afa967671a50f70445d473ae
Aug 12 06:59:06 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:06.097606 16923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:06.079955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:44861 (local address 127.26.226.62:39023)
0812 06:59:06.080476 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:06.080480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:06.081271 (+ 791us) server_negotiation.cc:407] Connection header received
0812 06:59:06.083085 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:06.083090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:06.083198 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:06.083362 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:06.085050 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:06.085826 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:06.087608 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:06.088017 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:06.088219 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:06.091665 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:06.091671 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:06.091674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:06.091706 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:06.094454 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:06.095197 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:06.095204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:06.095208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:06.095338 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:06.095796 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:06.095804 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:06.095808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:06.096169 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:06.096326 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:06.096996 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 06:59:06.097235 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":293,"threads_started":1}
I20260812 06:59:06.098526 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:06.080400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:39023 (local address 127.26.226.3:44861)
0812 06:59:06.081104 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:06.081146 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:06.082809 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:06.083600 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:06.083608 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:06.084156 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:06.084836 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:06.084873 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:06.085977 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:06.085982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:06.087447 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:06.087460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:06.088298 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:06.088305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:06.088562 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:06.089257 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:06.089279 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:06.091440 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:06.094602 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:06.094612 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:06.094640 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:06.095063 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:06.095453 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:06.095458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:06.095462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:06.095675 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:06.096298 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:06.096306 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:06.096633 (+ 327us) client_negotiation.cc:769] Sending connection context
0812 06:59:06.096911 (+ 278us) client_negotiation.cc:240] Negotiation successful
0812 06:59:06.097419 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":290,"threads_started":1}
I20260812 06:59:06.099897 16921 heartbeater.cc:347] Connected to a master server at 127.26.226.62:39023
I20260812 06:59:06.100253 16921 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:06.101049 16921 heartbeater.cc:511] Master 127.26.226.62:39023 requested a full tablet report, sending...
I20260812 06:59:06.102829 16134 ts_manager.cc:194] Registered new tserver with Master: d548df9f89be4ee788645e1e43d2d574 (127.26.226.3:39695)
I20260812 06:59:06.103737 16134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:44861
I20260812 06:59:06.114895 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:06.116614 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 16271
Aug 12 06:59:06 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:59:06.133567 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:06.117512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46244 (local address 127.26.226.1:39065)
0812 06:59:06.118328 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:06.118334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:06.118358 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:59:06.118415 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:06.118421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:06.118607 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:06.118779 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:06.120555 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:06.121951 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:06.123305 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:06.123848 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:06.123946 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:06.128009 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:06.128020 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:06.128034 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:06.128138 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:06.130991 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:06.131662 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:06.131669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:06.131671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:06.131808 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:06.132147 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:06.132151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:06.132153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:06.132496 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:06.132736 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:06.133057 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 06:59:06.133293 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":466,"threads_started":1}
I20260812 06:59:06.134822 16479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:06.218719 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 16271
I20260812 06:59:06.251278 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 16496
Aug 12 06:59:06 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:59:06.272821 16929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:06.252302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38982 (local address 127.26.226.2:37821)
0812 06:59:06.252895 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:06.252902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:06.252927 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:59:06.253005 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:06.253012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:06.253305 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:06.253480 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:06.254874 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:06.255893 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:06.257473 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:06.258072 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:06.258224 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:06.266649 (+ 8425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:06.266667 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:06.266687 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:06.266740 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:06.269726 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:06.270561 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:06.270569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:06.270571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:06.270742 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:06.271156 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:06.271160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:06.271162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:06.271533 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:06.271769 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:06.272157 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 06:59:06.272390 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":315,"threads_started":1}
I20260812 06:59:06.274196 16701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:06.361271 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 16496
I20260812 06:59:06.392469 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 16720
Aug 12 06:59:06 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:59:06.414536 16933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:06.393741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.26.226.3:39695)
0812 06:59:06.394577 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:06.394585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:06.394618 (+ 33us) server_negotiation.cc:407] Connection header received
0812 06:59:06.394685 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:06.394692 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:06.394947 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:06.395126 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:06.396477 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:06.397527 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:06.398958 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:06.399547 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:06.399650 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:06.405529 (+ 5879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:06.405560 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:06.405587 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:06.405642 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:06.410293 (+ 4651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:06.411488 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:06.411502 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:06.411508 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:06.411697 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:06.412182 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:06.412188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:06.412191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:06.412654 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:06.413017 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:06.413734 (+ 717us) server_negotiation.cc:299] Negotiation successful
0812 06:59:06.414064 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":499,"threads_started":1}
I20260812 06:59:06.416206 16911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:06.513554 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 16720
I20260812 06:59:06.537840 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 16083
I20260812 06:59:06.539105 16194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:06.649399 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 16083
I20260812 06:59:06.683779 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:39023
--webserver_interface=127.26.226.62
--webserver_port=43789
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:39023
--ranger_config_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:56507/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.226.20:56507/kms with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:07.030221 16939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:07.030537 16939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:07.030707 16939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:07.038249 16939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:07.038388 16939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:07.038496 16939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:07.038596 16939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:07.038690 16939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:07.049327 16939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.226.20:56507/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:39023
--ranger_config_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:39023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=43789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:07.050657 16939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:07.052335 16939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:07.069479 16955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:07.069558 16951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:07.069558 16948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:07.070858 16939 server_base.cc:1034] running on GCE node
I20260812 06:59:07.071729 16939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:07.073565 16939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:07.074808 16939 hybrid_clock.cc:648] HybridClock initialized: now 1786517947074750 us; error 51 us; skew 500 ppm
I20260812 06:59:07.075186 16939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:07 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517947, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:07.078413 16939 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:59:07.079913 16939 webserver.cc:460] Webserver started at http://127.26.226.62:43789/ using document root <none> and password file <none>
I20260812 06:59:07.080643 16939 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:07.080757 16939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:59:07 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517947, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:59:07.168879 16939 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:59:07.171973 16969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:07.173463 16939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:59:07.173672 16939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "a502e373644146d1a5a51aa2db8ebc1d"
format_stamp: "Formatted at 2026-08-12 06:59:02 on dist-test-slave-t3q3"
server_key: "8cec1205295e5b4372df237efe1a4904"
server_key_iv: "66778fc1357193fb38e0b0f62c901824"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:07.174646 16939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:07.201520 16939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:07.205830 16939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:07.206177 16939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:07.225394 16939 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:39023
I20260812 06:59:07.225412 17057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:39023 every 8 connection(s)
I20260812 06:59:07.226992 16939 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:07.234025 17059 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:59:07.237205 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 16939
I20260812 06:59:07.237777 17059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: Bootstrap starting.
I20260812 06:59:07.238399 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:56507/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:39065
--local_ip_for_outbound_sockets=127.26.226.1
--tserver_master_addrs=127.26.226.62:39023
--webserver_port=39233
--webserver_interface=127.26.226.1
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.226.20:56507/kms with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:07.260927 17059 log.cc:824] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:07.270084 17059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:59:07.270653 17059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: Bootstrap complete.
I20260812 06:59:07.280224 17059 raft_consensus.cc:348] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } }
I20260812 06:59:07.281271 17059 raft_consensus.cc:729] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a502e373644146d1a5a51aa2db8ebc1d, State: Initialized, Role: FOLLOWER
I20260812 06:59:07.281919 17059 consensus_queue.cc:260] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } }
I20260812 06:59:07.282168 17059 raft_consensus.cc:388] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:07.282296 17059 raft_consensus.cc:482] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:07.282491 17059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:59:07.288447 17059 raft_consensus.cc:504] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } }
I20260812 06:59:07.289070 17059 leader_election.cc:302] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a502e373644146d1a5a51aa2db8ebc1d; no voters:
I20260812 06:59:07.289793 17059 leader_election.cc:288] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:59:07.290092 17072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:59:07.303512 17072 raft_consensus.cc:686] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [term 2 LEADER]: Becoming Leader. State: Replica: a502e373644146d1a5a51aa2db8ebc1d, State: Running, Role: LEADER
I20260812 06:59:07.304629 17059 sys_catalog.cc:564] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:07.304437 17072 consensus_queue.cc:237] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } }
I20260812 06:59:07.315160 17073 sys_catalog.cc:455] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a502e373644146d1a5a51aa2db8ebc1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } } }
I20260812 06:59:07.315671 17073 sys_catalog.cc:458] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:07.317374 17074 sys_catalog.cc:455] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a502e373644146d1a5a51aa2db8ebc1d. Latest consensus state: current_term: 2 leader_uuid: "a502e373644146d1a5a51aa2db8ebc1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a502e373644146d1a5a51aa2db8ebc1d" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39023 } } }
I20260812 06:59:07.317777 17074 sys_catalog.cc:458] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:07.323271 17059 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 06:59:07.333954 17087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:07.342152 17087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:07.344147 17087 catalog_manager.cc:1259] Loaded cluster ID: 74d519394b6940edbcb1baeb3d38e24c
I20260812 06:59:07.344354 17087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:07.349000 17087 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:07.350931 17087 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: Loaded TSK: 0
I20260812 06:59:07.352986 17087 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:59:07.716832 17064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:07.717293 17064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:07.717496 17064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:07.729915 17064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:07.730096 17064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:07.730319 17064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:59:07.746735 17064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.226.20:56507/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:39065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=39233
--tserver_master_addrs=127.26.226.62:39023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:07.748684 17064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:07.750409 17064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:07.771960 17148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:07.776759 17153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:07.781155 17145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:07.782912 17064 server_base.cc:1034] running on GCE node
I20260812 06:59:07.783939 17064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:07.791796 17064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:07.793206 17064 hybrid_clock.cc:648] HybridClock initialized: now 1786517947793106 us; error 103 us; skew 500 ppm
I20260812 06:59:07.793676 17064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:07 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517947, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:07.798652 17064 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:59:07.801783 17064 webserver.cc:460] Webserver started at http://127.26.226.1:39233/ using document root <none> and password file <none>
I20260812 06:59:07.802793 17064 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:07.802964 17064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:59:07 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517947, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:59:07.846390 17064 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.002s sys 0.011s
I20260812 06:59:07.854334 17167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:07.860323 17064 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.002s sys 0.000s
I20260812 06:59:07.860569 17064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "78d93c67bd7049139125897dfbea521a"
format_stamp: "Formatted at 2026-08-12 06:59:04 on dist-test-slave-t3q3"
server_key: "fb251ab1c8f09b6f8723b9da55e30268"
server_key_iv: "2c4f9ae2e34cbced38ca78c0ee20b6dc"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:07.862114 17064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:07.909708 17064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:07.914409 17064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:07.914728 17064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:07.915889 17064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:07.917599 17064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:07.917732 17064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:07.917851 17064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:07.917935 17064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.004732 17064 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:39065
I20260812 06:59:08.004769 17367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:39065 every 8 connection(s)
I20260812 06:59:08.006623 17064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:08.012053 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 17064
I20260812 06:59:08.013002 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:56507/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:37821
--local_ip_for_outbound_sockets=127.26.226.2
--tserver_master_addrs=127.26.226.62:39023
--webserver_port=34089
--webserver_interface=127.26.226.2
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.226.20:56507/kms with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:59:08 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517947, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:08.043592 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.010149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:39023 (local address 127.26.226.1:50315)
0812 06:59:08.011039 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.011090 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:08.014569 (+ 3479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:08.017788 (+ 3219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:08.017800 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:08.018466 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:08.019157 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.019178 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.021441 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.021445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.029209 (+ 7764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.029240 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.030966 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.030974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.031267 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.032037 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:08.032118 (+ 81us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:08.034450 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:08.038488 (+ 4038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:08.038498 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:08.038517 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:08.039010 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:08.039544 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:08.039549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:08.039553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:08.039788 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:08.040785 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:08.040793 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:08.041173 (+ 380us) client_negotiation.cc:769] Sending connection context
0812 06:59:08.041469 (+ 296us) client_negotiation.cc:240] Negotiation successful
0812 06:59:08.042107 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":391,"threads_started":1}
I20260812 06:59:08.043598 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.010149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:50315 (local address 127.26.226.62:39023)
0812 06:59:08.015618 (+ 5469us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.015640 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.015681 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:59:08.016108 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.016133 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.016547 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.016997 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:08.019390 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.021212 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.029471 (+ 8259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.030669 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.030924 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.034810 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:08.034819 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:08.034837 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:08.034885 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:08.038282 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.039207 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.039215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.039221 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.039392 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.039942 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.039946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.039948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.040610 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:08.040833 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.041729 (+ 896us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.042122 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5195,"thread_start_us":4975,"threads_started":1}
I20260812 06:59:08.047104 17368 heartbeater.cc:347] Connected to a master server at 127.26.226.62:39023
I20260812 06:59:08.047492 17368 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:08.048395 17368 heartbeater.cc:511] Master 127.26.226.62:39023 requested a full tablet report, sending...
W20260812 06:59:08.051644 17368 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:39023 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 06:59:08.051896 17368 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:08.052238 17368 heartbeater.cc:511] Master 127.26.226.62:39023 requested a full tablet report, sending...
I20260812 06:59:08.053530 17368 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:08.053846 17368 heartbeater.cc:511] Master 127.26.226.62:39023 requested a full tablet report, sending...
W20260812 06:59:08.055155 17368 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:59:08.508286 17379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:08.508620 17379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:08.508785 17379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:08.519678 17379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:08.519830 17379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:08.520016 17379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:59:08.534705 17379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.226.20:56507/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:37821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=34089
--tserver_master_addrs=127.26.226.62:39023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:08.536434 17379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:08.538179 17379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:08.557147 17407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:08.557969 17403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:08.559165 17411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:08.560087 17379 server_base.cc:1034] running on GCE node
I20260812 06:59:08.560896 17379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:08.568974 17379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:08.572439 17379 hybrid_clock.cc:648] HybridClock initialized: now 1786517948572331 us; error 100 us; skew 500 ppm
I20260812 06:59:08.572885 17379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:08 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:08.577512 17379 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:59:08.587321 17379 webserver.cc:460] Webserver started at http://127.26.226.2:34089/ using document root <none> and password file <none>
I20260812 06:59:08.588537 17379 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:08.588742 17379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:59:08 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:59:08.625619 17379 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 06:59:08.628845 17429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.630395 17379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:59:08.630610 17379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "88c9932baa8f4b2bb823c1e2912d374f"
format_stamp: "Formatted at 2026-08-12 06:59:05 on dist-test-slave-t3q3"
server_key: "3b77e77a5d2598fd5610c5354b98e04f"
server_key_iv: "f56c1bdad17212ba21b0e329a5aff08a"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:08.631651 17379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:08.650781 17379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:08.654964 17379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:08.655260 17379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:08.656441 17379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:08.658512 17379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:08.658658 17379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.658802 17379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:08.658907 17379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:59:08 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:08.756728 17379 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:37821
I20260812 06:59:08.756786 17617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:37821 every 8 connection(s)
I20260812 06:59:08.758635 17379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:08.767119 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 17379
I20260812 06:59:08.767952 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.226.20:56507/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:39695
--local_ip_for_outbound_sockets=127.26.226.3
--tserver_master_addrs=127.26.226.62:39023
--webserver_port=42435
--webserver_interface=127.26.226.3
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.226.20:56507/kms with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:59:08 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:08.789932 17622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.763927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:56945 (local address 127.26.226.62:39023)
0812 06:59:08.765650 (+ 1723us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.765657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.765682 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:59:08.769853 (+ 4171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.769860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.769963 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.770067 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:08.772463 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.773749 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.776249 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.776991 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.777131 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.782017 (+ 4886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:08.782025 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:08.782030 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:08.782075 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:08.785745 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.786787 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.786793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.786797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.786950 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.787551 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.787558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.787564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.787925 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:08.788189 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.789269 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.789515 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1306,"thread_start_us":1190,"threads_started":1}
I20260812 06:59:08.792572 17621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.763117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:39023 (local address 127.26.226.2:56945)
0812 06:59:08.764258 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.764330 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:08.769372 (+ 5042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:08.770672 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:08.770685 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:08.771364 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:08.772234 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.772262 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.773963 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.773969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.776035 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.776051 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.777463 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.777471 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.777787 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.778628 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:08.778662 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:08.781700 (+ 3038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:08.785946 (+ 4246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:08.785955 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:08.785975 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:08.786633 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:08.787124 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:08.787129 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:08.787133 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:08.787409 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:08.788302 (+ 893us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:08.788311 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:08.788779 (+ 468us) client_negotiation.cc:769] Sending connection context
0812 06:59:08.790456 (+ 1677us) client_negotiation.cc:240] Negotiation successful
0812 06:59:08.791078 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":832,"threads_started":1}
I20260812 06:59:08.797458 17618 heartbeater.cc:347] Connected to a master server at 127.26.226.62:39023
I20260812 06:59:08.797885 17618 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:08.799005 17618 heartbeater.cc:511] Master 127.26.226.62:39023 requested a full tablet report, sending...
I20260812 06:59:08.801321 17000 ts_manager.cc:194] Registered new tserver with Master: 88c9932baa8f4b2bb823c1e2912d374f (127.26.226.2:37821)
I20260812 06:59:08.803902 17000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:56945
I20260812 06:59:09.055323 17368 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:09.055850 17368 heartbeater.cc:511] Master 127.26.226.62:39023 requested a full tablet report, sending...
I20260812 06:59:09.057217 17000 ts_manager.cc:194] Registered new tserver with Master: 78d93c67bd7049139125897dfbea521a (127.26.226.1:39065)
I20260812 06:59:09.058434 17000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:50315
Aug 12 06:59:09 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
W20260812 06:59:09.258873 17625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:09.259315 17625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:09.259526 17625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:09.270947 17625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:09.271107 17625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:09.271358 17625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:59:09.287366 17625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.226.20:56507/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:39695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=42435
--tserver_master_addrs=127.26.226.62:39023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:09.289201 17625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:09.290961 17625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:09.307930 17636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:09.308467 17634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:09.315794 17625 server_base.cc:1034] running on GCE node
W20260812 06:59:09.316111 17641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:09.316948 17625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:09.318645 17625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:09.319937 17625 hybrid_clock.cc:648] HybridClock initialized: now 1786517949319865 us; error 59 us; skew 500 ppm
I20260812 06:59:09.320354 17625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:09 dist-test-slave-t3q3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517949, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:09.324600 17625 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:59:09.326661 17625 webserver.cc:460] Webserver started at http://127.26.226.3:42435/ using document root <none> and password file <none>
I20260812 06:59:09.327661 17625 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:09.327839 17625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:59:09 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517949, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:59:09.357479 17625 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.006s sys 0.001s
I20260812 06:59:09.361420 17653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.363291 17625 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 06:59:09.363494 17625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "d548df9f89be4ee788645e1e43d2d574"
format_stamp: "Formatted at 2026-08-12 06:59:05 on dist-test-slave-t3q3"
server_key: "7582820385834d4d307add2e6ffe5984"
server_key_iv: "b3b64aab762634865e37996ce5dfadfa"
server_key_version: "kuduclusterkey@0"
I20260812 06:59:09.365010 17625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:09.424777 17625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:09.430671 17625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:09.431092 17625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:09.435155 17625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:09.438622 17625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:09.438831 17625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 06:59:09.439010 17625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:09.439162 17625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:59:09 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for HTTP/127.26.226.20@KRBTEST.COM
I20260812 06:59:09.653813 17625 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:39695
I20260812 06:59:09.653862 17835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:39695 every 8 connection(s)
I20260812 06:59:09.656270 17625 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:09.663527 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 17625
Aug 12 06:59:09 dist-test-slave-t3q3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517949, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:09.704303 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.664850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57296 (local address 127.26.226.62:39023)
0812 06:59:09.683376 (+ 18526us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.683386 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.684660 (+ 1274us) server_negotiation.cc:407] Connection header received
0812 06:59:09.684759 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.684767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.684898 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.685038 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:09.687787 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.688927 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.690937 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.691614 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.691736 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.695904 (+ 4168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:09.695912 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:09.695921 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:09.695965 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:09.699731 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:09.700670 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:09.700676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:09.700679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:09.700834 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:09.701679 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:09.701684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:09.701687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:09.702068 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:09.702334 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.703652 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.703881 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18342,"thread_start_us":18167,"threads_started":1}
I20260812 06:59:09.712100 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.666683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:48985 (local address 127.26.226.62:39023)
0812 06:59:09.687746 (+ 21063us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.687752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.689872 (+ 2120us) server_negotiation.cc:407] Connection header received
0812 06:59:09.692524 (+ 2652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.692530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.692638 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.692726 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:09.695447 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.696742 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.699376 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.700134 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.700225 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.704739 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:09.704747 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:09.704751 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:09.704798 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:09.708842 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:09.709629 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:09.709637 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:09.709644 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:09.709764 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:09.710221 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:09.710226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:09.710230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:09.710475 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:09.710624 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.711457 (+ 833us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.711676 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20909,"thread_start_us":21069,"threads_started":1}
I20260812 06:59:09.713336 17840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.665951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:39023 (local address 127.26.226.3:48985)
0812 06:59:09.688510 (+ 22559us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.688570 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.692189 (+ 3619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.693525 (+ 1336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.693540 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.694267 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:09.695202 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.695229 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.696958 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.696964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.699191 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.699218 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.700593 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.700603 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.700913 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.701662 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:09.701695 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:09.704449 (+ 2754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:09.709008 (+ 4559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:09.709016 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:09.709032 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:09.709497 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:09.709893 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:09.709897 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:09.709901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:09.710103 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:09.710615 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:09.710622 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:09.710939 (+ 317us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.711357 (+ 418us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.711934 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22173,"thread_start_us":22826,"threads_started":1}
I20260812 06:59:09.714844 17836 heartbeater.cc:347] Connected to a master server at 127.26.226.62:39023
I20260812 06:59:09.715178 17836 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:09.716051 17836 heartbeater.cc:511] Master 127.26.226.62:39023 requested a full tablet report, sending...
I20260812 06:59:09.717859 17000 ts_manager.cc:194] Registered new tserver with Master: d548df9f89be4ee788645e1e43d2d574 (127.26.226.3:39695)
I20260812 06:59:09.718940 17000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:48985
I20260812 06:59:09.724308 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:09.745751 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.727906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.26.226.62:39023)
0812 06:59:09.728116 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.728122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.728706 (+ 584us) server_negotiation.cc:407] Connection header received
0812 06:59:09.728890 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.728896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.728979 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.729097 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:09.731165 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.732315 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.734382 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.735011 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.735162 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.738755 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:09.738763 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:09.738768 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:09.738812 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:09.742350 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:09.743251 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:09.743258 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:09.743262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:09.743423 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:09.743915 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:09.743920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:09.743924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:09.744329 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:09.744538 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.745148 (+ 610us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.745405 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:59:09.752708 17000 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:59:09.763389 17000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:59:09.809199 17618 heartbeater.cc:503] Master 127.26.226.62:39023 was elected leader, sending a full tablet report...
I20260812 06:59:09.818291 17866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.795857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:37821 (local address 127.0.0.1:57860)
0812 06:59:09.796860 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.796914 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.797114 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.800450 (+ 3336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.800459 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.800495 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:09.800953 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.800971 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.803744 (+ 2773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.803753 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.807028 (+ 3275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.807058 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.816875 (+ 9817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.816885 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.817238 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.817321 (+ 83us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.817466 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.817737 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":610,"threads_started":1}
I20260812 06:59:09.820545 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.796606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57860 (local address 127.26.226.2:37821)
0812 06:59:09.797649 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.797656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.797713 (+ 57us) server_negotiation.cc:407] Connection header received
0812 06:59:09.797824 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.797835 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.798099 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.801188 (+ 3089us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:09.801480 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.803535 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.807307 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.809416 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.819329 (+ 9913us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.819697 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.819793 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.820041 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":902,"thread_start_us":766,"threads_started":1}
I20260812 06:59:09.826169 17508 tablet_service.cc:1467] Processing CreateTablet for tablet 721b79c5f95845f5a5f6454732bd3768 (DEFAULT_TABLE table=test-table [id=039c108b60d6464d962ff5fa73598444]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:09.827380 17870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.795952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:39695 (local address 127.0.0.1:33538)
0812 06:59:09.798067 (+ 2115us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.798085 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.798201 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.811014 (+ 12813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.811022 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.811075 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:09.811623 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.811639 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.822128 (+ 10489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.822136 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.824179 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.824197 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.826433 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.826440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.826614 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.826647 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.826747 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.826912 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1993,"mutex_wait_us":3,"thread_start_us":1865,"threads_started":1}
I20260812 06:59:09.827980 17508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 721b79c5f95845f5a5f6454732bd3768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:09.828680 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.797131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33538 (local address 127.26.226.3:39695)
0812 06:59:09.810201 (+ 13070us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.810214 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.810256 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:59:09.810385 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.810397 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.810728 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.811913 (+ 1185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:09.814393 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.816501 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.824393 (+ 7892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.825867 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.825998 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.826257 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.828177 (+ 1920us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.828359 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12807,"thread_start_us":14836,"threads_started":1}
I20260812 06:59:09.830114 17866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.801422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:39065 (local address 127.0.0.1:41606)
0812 06:59:09.818939 (+ 17517us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.818966 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.819106 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.820915 (+ 1809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.820923 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.820962 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:09.821194 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.821203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.825574 (+ 4371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.825579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.827490 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.827516 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.829404 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.829410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.829602 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.829635 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.829705 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.829854 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17353,"thread_start_us":19665,"threads_started":1}
I20260812 06:59:09.831171 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.801650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.26.226.1:39065)
0812 06:59:09.819760 (+ 18110us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.819770 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.819805 (+ 35us) server_negotiation.cc:407] Connection header received
0812 06:59:09.819912 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.819921 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.820525 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.820735 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:09.821397 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.825378 (+ 3981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.827728 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.829206 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.830354 (+ 1148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.830594 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.830671 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.830839 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17912,"thread_start_us":18440,"threads_started":1}
I20260812 06:59:09.832118 17734 tablet_service.cc:1467] Processing CreateTablet for tablet 721b79c5f95845f5a5f6454732bd3768 (DEFAULT_TABLE table=test-table [id=039c108b60d6464d962ff5fa73598444]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:09.833667 17734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 721b79c5f95845f5a5f6454732bd3768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:09.835606 17248 tablet_service.cc:1467] Processing CreateTablet for tablet 721b79c5f95845f5a5f6454732bd3768 (DEFAULT_TABLE table=test-table [id=039c108b60d6464d962ff5fa73598444]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:09.836978 17248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 721b79c5f95845f5a5f6454732bd3768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:09.852440 17880 tablet_bootstrap.cc:492] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: Bootstrap starting.
I20260812 06:59:09.854528 17881 tablet_bootstrap.cc:492] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: Bootstrap starting.
I20260812 06:59:09.858587 17881 tablet_bootstrap.cc:654] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:09.859576 17880 tablet_bootstrap.cc:654] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:09.860178 17881 log.cc:824] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:09.860922 17880 log.cc:824] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:09.868865 17884 tablet_bootstrap.cc:492] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a: Bootstrap starting.
I20260812 06:59:09.871155 17881 tablet_bootstrap.cc:492] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: No bootstrap required, opened a new log
I20260812 06:59:09.871527 17881 ts_tablet_manager.cc:1397] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20260812 06:59:09.871871 17880 tablet_bootstrap.cc:492] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: No bootstrap required, opened a new log
I20260812 06:59:09.872195 17880 ts_tablet_manager.cc:1397] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.007s
I20260812 06:59:09.873287 17884 tablet_bootstrap.cc:654] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:09.874682 17884 log.cc:824] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:09.882627 17881 raft_consensus.cc:348] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c9932baa8f4b2bb823c1e2912d374f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 37821 } } peers { permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 } } peers { permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 } }
I20260812 06:59:09.883282 17881 raft_consensus.cc:374] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:09.883478 17881 raft_consensus.cc:729] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d548df9f89be4ee788645e1e43d2d574, State: Initialized, Role: FOLLOWER
I20260812 06:59:09.884305 17884 tablet_bootstrap.cc:492] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a: No bootstrap required, opened a new log
I20260812 06:59:09.884370 17881 consensus_queue.cc:260] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c9932baa8f4b2bb823c1e2912d374f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 37821 } } peers { permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 } } peers { permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 } }
I20260812 06:59:09.884651 17884 ts_tablet_manager.cc:1397] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20260812 06:59:09.885331 17880 raft_consensus.cc:348] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c9932baa8f4b2bb823c1e2912d374f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 37821 } } peers { permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 } } peers { permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 } }
I20260812 06:59:09.885813 17880 raft_consensus.cc:374] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:09.885950 17880 raft_consensus.cc:729] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c9932baa8f4b2bb823c1e2912d374f, State: Initialized, Role: FOLLOWER
I20260812 06:59:09.886603 17880 consensus_queue.cc:260] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c9932baa8f4b2bb823c1e2912d374f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 37821 } } peers { permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 } } peers { permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 } }
I20260812 06:59:09.899521 17836 heartbeater.cc:503] Master 127.26.226.62:39023 was elected leader, sending a full tablet report...
I20260812 06:59:09.900414 17881 ts_tablet_manager.cc:1428] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: Time spent starting tablet: real 0.029s user 0.023s sys 0.004s
I20260812 06:59:09.900488 17880 ts_tablet_manager.cc:1428] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: Time spent starting tablet: real 0.028s user 0.018s sys 0.008s
I20260812 06:59:09.902362 17884 raft_consensus.cc:348] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c9932baa8f4b2bb823c1e2912d374f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 37821 } } peers { permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 } } peers { permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 } }
I20260812 06:59:09.902933 17884 raft_consensus.cc:374] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:09.903095 17884 raft_consensus.cc:729] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d93c67bd7049139125897dfbea521a, State: Initialized, Role: FOLLOWER
I20260812 06:59:09.903856 17884 consensus_queue.cc:260] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c9932baa8f4b2bb823c1e2912d374f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 37821 } } peers { permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 } } peers { permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 } }
I20260812 06:59:09.908174 17368 heartbeater.cc:503] Master 127.26.226.62:39023 was elected leader, sending a full tablet report...
I20260812 06:59:09.908807 17884 ts_tablet_manager.cc:1428] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a: Time spent starting tablet: real 0.024s user 0.012s sys 0.009s
W20260812 06:59:09.954077 17837 tablet.cc:2369] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:09.966230 17893 raft_consensus.cc:482] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:09.966588 17893 raft_consensus.cc:504] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c9932baa8f4b2bb823c1e2912d374f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 37821 } } peers { permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 } } peers { permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 } }
I20260812 06:59:09.968691 17893 leader_election.cc:288] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d548df9f89be4ee788645e1e43d2d574 (127.26.226.3:39695), 78d93c67bd7049139125897dfbea521a (127.26.226.1:39065)
I20260812 06:59:09.977530 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.969319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:56443 (local address 127.26.226.1:39065)
0812 06:59:09.969554 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.969560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.970647 (+ 1087us) server_negotiation.cc:407] Connection header received
0812 06:59:09.970904 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.970910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.970983 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.971078 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:09.971730 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.972547 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.975561 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.976419 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.976517 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.976569 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.977102 (+ 533us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.977263 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:59:09.977953 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.969665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:36543 (local address 127.26.226.3:39695)
0812 06:59:09.969866 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.969872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.970143 (+ 271us) server_negotiation.cc:407] Connection header received
0812 06:59:09.970510 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.970516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.970613 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.970680 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:09.971556 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.972674 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.975558 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.976781 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.976936 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.977005 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.977391 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.977522 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:59:09.978071 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.969161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:39695 (local address 127.26.226.2:36543)
0812 06:59:09.970029 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.970048 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.970292 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.970909 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.970913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.970938 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:09.971369 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.971388 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.972809 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.972815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.975382 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.975397 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.977066 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.977074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.977237 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.977266 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.977356 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.977475 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":117,"thread_start_us":1077,"threads_started":1}
I20260812 06:59:09.978371 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.969199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:39065 (local address 127.26.226.2:56443)
0812 06:59:09.970538 (+ 1339us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.970556 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.970675 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.971115 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.971124 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.971143 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:09.971556 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.971569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.972695 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.972701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.975386 (+ 2685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.975397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.976719 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.976724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.976867 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.976905 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.977739 (+ 834us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.977871 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1226,"mutex_wait_us":200,"thread_start_us":658,"threads_started":1}
I20260812 06:59:09.979358 17287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "721b79c5f95845f5a5f6454732bd3768" candidate_uuid: "88c9932baa8f4b2bb823c1e2912d374f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d93c67bd7049139125897dfbea521a" is_pre_election: true
I20260812 06:59:09.979337 17767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "721b79c5f95845f5a5f6454732bd3768" candidate_uuid: "88c9932baa8f4b2bb823c1e2912d374f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d548df9f89be4ee788645e1e43d2d574" is_pre_election: true
I20260812 06:59:09.979979 17767 raft_consensus.cc:2454] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88c9932baa8f4b2bb823c1e2912d374f in term 0.
I20260812 06:59:09.979979 17287 raft_consensus.cc:2454] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88c9932baa8f4b2bb823c1e2912d374f in term 0.
I20260812 06:59:09.980738 17431 leader_election.cc:302] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78d93c67bd7049139125897dfbea521a, 88c9932baa8f4b2bb823c1e2912d374f; no voters:
I20260812 06:59:09.981190 17893 raft_consensus.cc:2781] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:09.981345 17893 raft_consensus.cc:482] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:09.981455 17893 raft_consensus.cc:3037] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:09.983660 17893 raft_consensus.cc:504] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c9932baa8f4b2bb823c1e2912d374f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 37821 } } peers { permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 } } peers { permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 } }
I20260812 06:59:09.984368 17893 leader_election.cc:288] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [CANDIDATE]: Term 1 election: Requested vote from peers d548df9f89be4ee788645e1e43d2d574 (127.26.226.3:39695), 78d93c67bd7049139125897dfbea521a (127.26.226.1:39065)
I20260812 06:59:09.984754 17767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "721b79c5f95845f5a5f6454732bd3768" candidate_uuid: "88c9932baa8f4b2bb823c1e2912d374f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d548df9f89be4ee788645e1e43d2d574"
I20260812 06:59:09.984869 17287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "721b79c5f95845f5a5f6454732bd3768" candidate_uuid: "88c9932baa8f4b2bb823c1e2912d374f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d93c67bd7049139125897dfbea521a"
I20260812 06:59:09.984968 17767 raft_consensus.cc:3037] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:09.985078 17287 raft_consensus.cc:3037] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:09.987164 17767 raft_consensus.cc:2454] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88c9932baa8f4b2bb823c1e2912d374f in term 1.
I20260812 06:59:09.987558 17432 leader_election.cc:302] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88c9932baa8f4b2bb823c1e2912d374f, d548df9f89be4ee788645e1e43d2d574; no voters:
I20260812 06:59:09.987881 17893 raft_consensus.cc:2781] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:09.988193 17287 raft_consensus.cc:2454] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88c9932baa8f4b2bb823c1e2912d374f in term 1.
I20260812 06:59:09.988451 17893 raft_consensus.cc:686] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 1 LEADER]: Becoming Leader. State: Replica: 88c9932baa8f4b2bb823c1e2912d374f, State: Running, Role: LEADER
I20260812 06:59:09.989271 17893 consensus_queue.cc:237] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c9932baa8f4b2bb823c1e2912d374f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 37821 } } peers { permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 } } peers { permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 } }
I20260812 06:59:09.994253 17000 catalog_manager.cc:5495] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88c9932baa8f4b2bb823c1e2912d374f (127.26.226.2). New cstate: current_term: 1 leader_uuid: "88c9932baa8f4b2bb823c1e2912d374f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c9932baa8f4b2bb823c1e2912d374f" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 37821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 } health_report { overall_health: UNKNOWN } } }
W20260812 06:59:10.016593 17374 tablet.cc:2369] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:59:10.019946 17619 tablet.cc:2369] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:10.053113 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.047127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57872 (local address 127.26.226.2:37821)
0812 06:59:10.047435 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:10.047441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:10.047542 (+ 101us) server_negotiation.cc:407] Connection header received
0812 06:59:10.047806 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:10.047811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:10.047912 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:10.048133 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:10.048517 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.049345 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.050738 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.051202 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.051331 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:10.051899 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:10.052476 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:10.052589 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:10.052745 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 06:59:10.052813 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 06:59:10.061641 17767 raft_consensus.cc:1261] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574 [term 1 FOLLOWER]: Refusing update from remote peer 88c9932baa8f4b2bb823c1e2912d374f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:10.061770 17287 raft_consensus.cc:1261] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a [term 1 FOLLOWER]: Refusing update from remote peer 88c9932baa8f4b2bb823c1e2912d374f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:10.062767 17899 consensus_queue.cc:1035] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d548df9f89be4ee788645e1e43d2d574" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:10.063146 17893 consensus_queue.cc:1035] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d93c67bd7049139125897dfbea521a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:10.077917 17902 mvcc.cc:205] Tried to move back new op lower bound from 7317577523441774592 to 7317577523160215552. Current Snapshot: MvccSnapshot[applied={T|T < 7317577523441774592}]
I20260812 06:59:10.111955 17000 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:10.112346 17000 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:10.118467 17000 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a502e373644146d1a5a51aa2db8ebc1d: Sending DeleteTablet for 3 replicas of tablet 721b79c5f95845f5a5f6454732bd3768
I20260812 06:59:10.120257 17508 tablet_service.cc:1514] Processing DeleteTablet for tablet 721b79c5f95845f5a5f6454732bd3768 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:10 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:57860
I20260812 06:59:10.122212 17734 tablet_service.cc:1514] Processing DeleteTablet for tablet 721b79c5f95845f5a5f6454732bd3768 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:10 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:33538
I20260812 06:59:10.124889 17248 tablet_service.cc:1514] Processing DeleteTablet for tablet 721b79c5f95845f5a5f6454732bd3768 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:10 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:41606
I20260812 06:59:10.126520 17926 tablet_replica.cc:330] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a: stopping tablet replica
I20260812 06:59:10.129884 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 17064
I20260812 06:59:10.179842 17920 tablet_replica.cc:330] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: stopping tablet replica
I20260812 06:59:10.180713 17920 raft_consensus.cc:2229] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:59:10.181509 17920 raft_consensus.cc:2258] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:10.182660 17924 tablet_replica.cc:330] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: stopping tablet replica
I20260812 06:59:10.186604 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.134101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.26.226.1:39065)
0812 06:59:10.134355 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:10.134364 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:10.134392 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:59:10.134535 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:10.134543 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:10.134647 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:10.134775 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:10.137589 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.139160 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.143219 (+ 4059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.144206 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.144368 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:10.162644 (+ 18276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:10.162672 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:10.162702 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:10.162786 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:10.169513 (+ 6727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:10.173908 (+ 4395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:10.173924 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:10.173930 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:10.174190 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:10.177098 (+ 2908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:10.177106 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:10.177112 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:10.177907 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:10.178513 (+ 606us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:10.185556 (+ 7043us) server_negotiation.cc:299] Negotiation successful
0812 06:59:10.185999 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 06:59:10.187516 17926 raft_consensus.cc:2229] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:10.194453 17359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:10.196132 17924 raft_consensus.cc:2229] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:10.200047 17924 raft_consensus.cc:2258] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:10.200884 17920 ts_tablet_manager.cc:1905] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:10.202414 17924 ts_tablet_manager.cc:1905] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:10.209131 17920 ts_tablet_manager.cc:1918] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:10.209425 17920 log.cc:1196] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/721b79c5f95845f5a5f6454732bd3768
I20260812 06:59:10.209964 17924 ts_tablet_manager.cc:1918] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:10.210055 17920 ts_tablet_manager.cc:1939] T 721b79c5f95845f5a5f6454732bd3768 P 88c9932baa8f4b2bb823c1e2912d374f: Deleting consensus metadata
I20260812 06:59:10.210268 17924 log.cc:1196] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/721b79c5f95845f5a5f6454732bd3768
I20260812 06:59:10.210914 17924 ts_tablet_manager.cc:1939] T 721b79c5f95845f5a5f6454732bd3768 P d548df9f89be4ee788645e1e43d2d574: Deleting consensus metadata
I20260812 06:59:10.212190 16974 catalog_manager.cc:4841] TS 88c9932baa8f4b2bb823c1e2912d374f (127.26.226.2:37821): tablet 721b79c5f95845f5a5f6454732bd3768 (table test-table [id=039c108b60d6464d962ff5fa73598444]) successfully deleted
I20260812 06:59:10.213044 16978 catalog_manager.cc:4841] TS d548df9f89be4ee788645e1e43d2d574 (127.26.226.3:39695): tablet 721b79c5f95845f5a5f6454732bd3768 (table test-table [id=039c108b60d6464d962ff5fa73598444]) successfully deleted
I20260812 06:59:10.345002 17926 raft_consensus.cc:2258] T 721b79c5f95845f5a5f6454732bd3768 P 78d93c67bd7049139125897dfbea521a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:10.345999 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 17064
W20260812 06:59:10.369660 16974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.226.1:39065 (error 108)
I20260812 06:59:10.370048 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 17379
I20260812 06:59:10.372233 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.371377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:39065 (local address 127.0.0.1:41620)
0812 06:59:10.371858 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.371977 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:39065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":235,"threads_started":1}
W20260812 06:59:10.372537 16974 catalog_manager.cc:4584] TS 78d93c67bd7049139125897dfbea521a (127.26.226.1:39065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 721b79c5f95845f5a5f6454732bd3768: Network error: Client connection negotiation failed: client connection to 127.26.226.1:39065: connect: Connection refused (error 111)
I20260812 06:59:10.385952 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.371082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57874 (local address 127.26.226.2:37821)
0812 06:59:10.371296 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:10.371302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:10.371328 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:59:10.371425 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:10.371431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:10.371563 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:10.371685 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:10.373702 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.374585 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.376476 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.376969 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.377085 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:10.379948 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:10.379960 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:10.379975 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:10.380023 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:10.383187 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:10.383894 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:10.383910 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:10.383912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:10.384113 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:10.384545 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:10.384548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:10.384550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:10.384891 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:10.385150 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:10.385471 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 06:59:10.385658 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:59:10.386845 17608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:10.426391 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.425829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:39065 (local address 127.0.0.1:41636)
0812 06:59:10.426082 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.426211 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:39065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 06:59:10.427623 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.427179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:39065 (local address 127.0.0.1:41652)
0812 06:59:10.427349 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.427439 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:39065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 06:59:10.488391 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.487802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:39065 (local address 127.0.0.1:41664)
0812 06:59:10.488084 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.488206 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:39065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 06:59:10.489516 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.489115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:39065 (local address 127.0.0.1:41666)
0812 06:59:10.489269 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.489351 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:39065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 06:59:10.505030 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 17379
I20260812 06:59:10.537308 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 17625
I20260812 06:59:10.551822 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.538119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33550 (local address 127.26.226.3:39695)
0812 06:59:10.538654 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:10.538658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:10.538676 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:59:10.538733 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:10.538737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:10.538805 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:10.538903 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:10.540380 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.541092 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.542414 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.542820 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.542919 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:10.545824 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:10.545839 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:10.545856 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:10.545888 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:10.548959 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:10.549796 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:10.549804 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:10.549807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:10.549960 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:10.550327 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:10.550331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:10.550334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:10.550723 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:10.550942 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:10.551223 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 06:59:10.551392 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":291,"threads_started":1}
I20260812 06:59:10.552717 17826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:10.570082 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.569529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:39065 (local address 127.0.0.1:41672)
0812 06:59:10.569782 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.569894 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:39065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260812 06:59:10.571249 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.570759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:39065 (local address 127.0.0.1:41682)
0812 06:59:10.570986 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.571084 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:39065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260812 06:59:10.643610 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 17625
I20260812 06:59:10.670433 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 16939
I20260812 06:59:10.671679 17050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:10.788360 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 16939
I20260812 06:59:10.788405 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.787797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:39065 (local address 127.0.0.1:41698)
0812 06:59:10.788046 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.788186 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:39065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 06:59:10.825678 27528 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 06:59:11.171353 27528 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 06:59:11.171604 27528 mini_ranger.cc:66] Stopping Ranger...
I20260812 06:59:11.517835 27528 mini_ranger.cc:68] Stopped Ranger
2026-08-12 06:59:11.518 UTC [15352] LOG: received smart shutdown request
2026-08-12 06:59:11.521 UTC [15352] LOG: background worker "logical replication launcher" (PID 15363) exited with exit code 1
2026-08-12 06:59:11.521 UTC [15358] LOG: shutting down
2026-08-12 06:59:11.690 UTC [15352] LOG: database system is shut down
2026-08-12T06:59:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56324 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:11 dist-test-slave-t3q3 krb5kdc[17951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:11 dist-test-slave-t3q3 krb5kdc[17951](info): set up 2 sockets
Aug 12 06:59:11 dist-test-slave-t3q3 krb5kdc[17951](info): commencing operation
krb5kdc: starting...
W20260812 06:59:13.198704 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:59:13 dist-test-slave-t3q3 krb5kdc[17951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:59:13.508572 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34613
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:34613,127.26.226.61:46691,127.26.226.60:40699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:13.862555 17973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:13.862947 17973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:13.863109 17973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:13.870611 17973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:13.870776 17973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:13.870884 17973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:13.870986 17973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:13.871057 17973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:13.882093 17973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:34613,127.26.226.61:46691,127.26.226.60:40699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:13.883610 17973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:13.884980 17973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:13.898150 17985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:13.898205 17988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:13.898241 17991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:13.899021 17973 server_base.cc:1034] running on GCE node
I20260812 06:59:13.899890 17973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:13.901499 17973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:13.902781 17973 hybrid_clock.cc:648] HybridClock initialized: now 1786517953902712 us; error 63 us; skew 500 ppm
I20260812 06:59:13.903196 17973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:13 dist-test-slave-t3q3 krb5kdc[17951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517953, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:13.911118 17973 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:59:13.912906 17973 webserver.cc:460] Webserver started at http://127.26.226.62:39535/ using document root <none> and password file <none>
I20260812 06:59:13.913681 17973 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:13.913848 17973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:13.914201 17973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:13.917474 17973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "e2562adff7864bad866590ca850e5bc4"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-t3q3"
server_key: "a04bb64926f0879534e00fc1d3614803"
server_key_iv: "0491e51603290bbffeafffe758b500e6"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.918350 17973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "e2562adff7864bad866590ca850e5bc4"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-t3q3"
server_key: "a04bb64926f0879534e00fc1d3614803"
server_key_iv: "0491e51603290bbffeafffe758b500e6"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.923961 17973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 06:59:13.927592 18008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:13.929220 17973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 06:59:13.929380 17973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "e2562adff7864bad866590ca850e5bc4"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-t3q3"
server_key: "a04bb64926f0879534e00fc1d3614803"
server_key_iv: "0491e51603290bbffeafffe758b500e6"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.929581 17973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:13.972553 17973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:13.976948 17973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:13.977231 17973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:13.996192 17973 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:34613
I20260812 06:59:13.996207 18090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:34613 every 8 connection(s)
I20260812 06:59:13.997674 17973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:13.999647 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 17973
I20260812 06:59:13.999904 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:59:14.000686 27528 external_mini_cluster.cc:1428] Setting key 8a619c630cdaadbf1eca25ebf94b6229
I20260812 06:59:14.001706 18094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:14.008466 18094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:59:14 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517953, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.226.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.61@KRBTEST.COM" created.
I20260812 06:59:14.032294 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.011768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34613 (local address 127.0.0.1:47506)
0812 06:59:14.012873 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.012937 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:14.015068 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:14.015939 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:14.015946 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:14.016722 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:14.017049 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:14.017067 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.018629 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.018639 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:14.019707 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:14.019719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.020752 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.020759 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:14.021009 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:14.021739 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:14.021762 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:14.024461 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:14.027908 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:14.027924 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:14.027941 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:14.028593 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:14.029034 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:14.029038 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:14.029040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:14.029204 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:14.030084 (+ 880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:14.030093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:14.030283 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 06:59:14.030496 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 06:59:14.030898 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":3,"thread_start_us":467,"threads_started":1}
I20260812 06:59:14.032344 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.011769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47506 (local address 127.26.226.62:34613)
0812 06:59:14.013690 (+ 1921us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:14.013704 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:14.013773 (+ 69us) server_negotiation.cc:407] Connection header received
0812 06:59:14.015461 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:14.015467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:14.015701 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:14.015915 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:14.017257 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.018453 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.019852 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.020496 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.020703 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:14.024716 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:14.024724 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:14.024739 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:14.024768 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:14.027738 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:14.028750 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:14.028758 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:14.028760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:14.028923 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:14.029314 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:14.029317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:14.029319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:14.029954 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:14.030109 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:14.030691 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 06:59:14.031021 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":142,"server-negotiator.queue_time_us":1692,"spinlock_wait_cycles":93568,"thread_start_us":1432,"threads_started":1}
I20260812 06:59:14.035561 18094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:14.036803 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.036392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34572)
0812 06:59:14.036570 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.036688 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 06:59:14.037135 18011 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
I20260812 06:59:14.038829 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.038435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34588)
0812 06:59:14.038640 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.038705 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 06:59:14.039060 18094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:46691: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab.
Entry for principal kudu/127.26.226.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab.
I20260812 06:59:14.067646 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.61:46691
--webserver_interface=127.26.226.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab
--principal=kudu/127.26.226.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:34613,127.26.226.61:46691,127.26.226.60:40699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:14.088385 18094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } attempt: 1
I20260812 06:59:14.089805 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.089260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34594)
0812 06:59:14.089514 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.089643 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 06:59:14.091140 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.090606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34608)
0812 06:59:14.090804 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.090918 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 06:59:14.091430 18094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:46691: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
I20260812 06:59:14.159739 18094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } attempt: 2
I20260812 06:59:14.161226 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.160624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34614)
0812 06:59:14.160945 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.161077 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 06:59:14.162420 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.162060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34622)
0812 06:59:14.162198 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.162282 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:59:14.162688 18094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:46691: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
I20260812 06:59:14.253973 18094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } attempt: 3
I20260812 06:59:14.255328 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.254829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34624)
0812 06:59:14.255076 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.255192 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260812 06:59:14.256572 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.256185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34638)
0812 06:59:14.256341 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.256425 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 06:59:14.256784 18094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:46691: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
I20260812 06:59:14.400141 18094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } attempt: 4
I20260812 06:59:14.401362 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.400804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34654)
0812 06:59:14.401140 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.401226 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260812 06:59:14.402370 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.402018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34656)
0812 06:59:14.402165 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.402251 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 06:59:14.402554 18094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:46691: Network error: Client connection negotiation failed: client connection to 127.26.226.61:46691: connect: Connection refused (error 111)
W20260812 06:59:14.421058 18106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:14.421435 18106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:14.421599 18106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:14.428766 18106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:14.428916 18106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:14.429029 18106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:14.429145 18106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:14.429234 18106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:14.439205 18106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:34613,127.26.226.61:46691,127.26.226.60:40699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab
--principal=kudu/127.26.226.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.61:46691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.226.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:14.440547 18106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:14.441865 18106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:14.453708 18118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:14.453661 18121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:14.453675 18119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:14.454643 18106 server_base.cc:1034] running on GCE node
I20260812 06:59:14.455435 18106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:14.456910 18106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:14.458171 18106 hybrid_clock.cc:648] HybridClock initialized: now 1786517954458131 us; error 53 us; skew 500 ppm
I20260812 06:59:14.458634 18106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:14 dist-test-slave-t3q3 krb5kdc[17951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517954, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:14.462049 18106 init.cc:377] Logged in from keytab as kudu/127.26.226.61@KRBTEST.COM (short username kudu)
I20260812 06:59:14.463510 18106 webserver.cc:460] Webserver started at http://127.26.226.61:35745/ using document root <none> and password file <none>
I20260812 06:59:14.464270 18106 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:14.464422 18106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:14.464744 18106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:14.467718 18106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/data/instance:
uuid: "d391094ea5ef4cb8acd367121eac200d"
format_stamp: "Formatted at 2026-08-12 06:59:14 on dist-test-slave-t3q3"
server_key: "6c3a23e38a118350014621e7bce0694f"
server_key_iv: "3028a2ab86b04bbc4db15a00259d51bd"
server_key_version: "encryptionkey@0"
I20260812 06:59:14.468556 18106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/wal/instance:
uuid: "d391094ea5ef4cb8acd367121eac200d"
format_stamp: "Formatted at 2026-08-12 06:59:14 on dist-test-slave-t3q3"
server_key: "6c3a23e38a118350014621e7bce0694f"
server_key_iv: "3028a2ab86b04bbc4db15a00259d51bd"
server_key_version: "encryptionkey@0"
I20260812 06:59:14.473796 18106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:59:14.477361 18141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:14.478682 18106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:59:14.478876 18106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/wal
uuid: "d391094ea5ef4cb8acd367121eac200d"
format_stamp: "Formatted at 2026-08-12 06:59:14 on dist-test-slave-t3q3"
server_key: "6c3a23e38a118350014621e7bce0694f"
server_key_iv: "3028a2ab86b04bbc4db15a00259d51bd"
server_key_version: "encryptionkey@0"
I20260812 06:59:14.479085 18106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:14.493646 18106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:14.498306 18106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:14.498615 18106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:14.517400 18106 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.61:46691
I20260812 06:59:14.517436 18223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.61:46691 every 8 connection(s)
I20260812 06:59:14.518855 18106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
I20260812 06:59:14.522488 18225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:14.528193 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 18106
I20260812 06:59:14.528395 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-1/wal/instance
I20260812 06:59:14.528863 27528 external_mini_cluster.cc:1428] Setting key 461009c9a03ba97a2b6c0bcd96ca4365
I20260812 06:59:14.529265 18225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:59:14 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517954, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:14.553921 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.531878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.26.226.62:34613)
0812 06:59:14.532114 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:14.532122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:14.533849 (+ 1727us) server_negotiation.cc:407] Connection header received
0812 06:59:14.536244 (+ 2395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:14.536253 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:14.536350 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:14.536457 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:14.538370 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.539450 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.541737 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.542491 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.542635 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:14.547318 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:14.547323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:14.547326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:14.547360 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:14.550123 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:14.551215 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:14.551219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:14.551221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:14.551348 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:14.551952 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:14.551955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:14.551957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:14.552336 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:14.552454 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:14.553511 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0812 06:59:14.553697 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:59:14.555658 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.532493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34613 (local address 127.0.0.1:47508)
0812 06:59:14.533363 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.533407 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:14.535909 (+ 2502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:14.536695 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:14.536707 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:14.537415 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:14.538148 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:14.538172 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.539624 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.539631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:14.541546 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:14.541575 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.542770 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.542780 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:14.543122 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:14.543897 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:14.543932 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:14.547058 (+ 3126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:14.550285 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:14.550297 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:14.550318 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:14.551070 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:14.551504 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:14.551530 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:14.551541 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:14.551828 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:14.552492 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:14.552505 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:14.553089 (+ 584us) client_negotiation.cc:769] Sending connection context
0812 06:59:14.553416 (+ 327us) client_negotiation.cc:240] Negotiation successful
0812 06:59:14.554033 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":415,"threads_started":1}
I20260812 06:59:14.557603 18225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.226.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.60@KRBTEST.COM" created.
Aug 12 06:59:14 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517954, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 06:59:14.574817 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.558449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34672)
0812 06:59:14.558767 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.558791 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:14.558914 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:14.559950 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:14.559956 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:14.560534 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:14.560695 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:14.560704 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.562138 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.562142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:14.563418 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:14.563431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.564724 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.564728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:14.564887 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:14.565534 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:14.565551 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:14.567711 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:14.572188 (+ 4477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:14.572192 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:14.572195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:14.572559 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:14.572990 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:14.572996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:14.573002 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:14.573155 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:14.573792 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:14.573797 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:14.573946 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 06:59:14.574091 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 06:59:14.574378 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":2}
I20260812 06:59:14.574890 18230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.558758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.26.226.61:46691)
0812 06:59:14.559495 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:14.559501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:14.559527 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:59:14.559574 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:14.559579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:14.559768 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:14.559891 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:14.560849 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.562002 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.563561 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.564547 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.564726 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:14.567950 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:14.567966 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:14.567987 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:14.568032 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:14.572021 (+ 3989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:14.572730 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:14.572739 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:14.572749 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:14.572863 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:14.573263 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:14.573270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:14.573273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:14.573638 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:14.573845 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:14.574233 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 06:59:14.574487 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":584,"spinlock_wait_cycles":26496,"thread_start_us":445,"threads_started":1}
I20260812 06:59:14.577832 18225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:14.579057 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.578569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33498)
0812 06:59:14.578769 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.578897 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260812 06:59:14.579310 18144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
I20260812 06:59:14.581023 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.580657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33504)
0812 06:59:14.580840 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.580913 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:59:14.581272 18225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:40699: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab.
Entry for principal kudu/127.26.226.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab.
I20260812 06:59:14.600973 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.60:40699
--webserver_interface=127.26.226.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab
--principal=kudu/127.26.226.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:34613,127.26.226.61:46691,127.26.226.60:40699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:14.630589 18225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } attempt: 1
I20260812 06:59:14.631899 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.631331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33520)
0812 06:59:14.631574 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.631709 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260812 06:59:14.633037 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.632626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33534)
0812 06:59:14.632795 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.632884 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 06:59:14.633273 18225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:40699: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
I20260812 06:59:14.701540 18225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } attempt: 2
I20260812 06:59:14.701943 18094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } attempt: 5
I20260812 06:59:14.702950 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.702359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33538)
0812 06:59:14.702620 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.702739 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260812 06:59:14.704116 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.703636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33548)
0812 06:59:14.703842 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.703925 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260812 06:59:14.704378 18225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:40699: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Aug 12 06:59:14 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517953, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 06:59:14.716369 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.702686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34676)
0812 06:59:14.702973 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.702994 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:14.703138 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:14.703605 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:14.703611 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:14.704180 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:14.704443 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:14.704454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.705461 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.705468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:14.706343 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:14.706354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.707124 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.707130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:14.707309 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:14.708108 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:14.708144 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:14.710426 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:14.713680 (+ 3254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:14.713686 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:14.713691 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:14.714198 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:14.714587 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:14.714592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:14.714596 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:14.714745 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:14.715319 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:14.715325 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:14.715507 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 06:59:14.715657 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 06:59:14.715959 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260812 06:59:14.716369 18230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.702753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34676 (local address 127.26.226.61:46691)
0812 06:59:14.702981 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:14.702991 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:14.703180 (+ 189us) server_negotiation.cc:407] Connection header received
0812 06:59:14.703289 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:14.703296 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:14.703393 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:14.703507 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:14.704611 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.705320 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.706487 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:14.706951 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:14.707096 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:14.710633 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:14.710638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:14.710641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:14.710672 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:14.713521 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:14.714326 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:14.714333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:14.714337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:14.714465 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:14.714862 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:14.714868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:14.714871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:14.715189 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:14.715353 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:14.715773 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 06:59:14.716000 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:59:14.717519 18094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:14.718727 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.718195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33554)
0812 06:59:14.718420 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.718530 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 06:59:14.719808 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.719332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33566)
0812 06:59:14.719545 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.719661 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 06:59:14.720041 18094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:40699: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
I20260812 06:59:14.771347 18094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } attempt: 1
I20260812 06:59:14.772953 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.772249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33576)
0812 06:59:14.772517 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.772693 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 06:59:14.774163 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.773731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33578)
0812 06:59:14.773927 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.774023 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 06:59:14.774432 18094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:40699: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
I20260812 06:59:14.795665 18225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } attempt: 3
I20260812 06:59:14.796936 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.796408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33592)
0812 06:59:14.796645 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.796752 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 06:59:14.798137 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.797688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33606)
0812 06:59:14.797941 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.798018 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260812 06:59:14.798357 18225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:40699: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
I20260812 06:59:14.842692 18094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } attempt: 2
I20260812 06:59:14.843971 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.843422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33616)
0812 06:59:14.843675 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.843772 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 06:59:14.845201 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.844870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33618)
0812 06:59:14.845021 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.845094 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 06:59:14.845504 18094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:40699: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
I20260812 06:59:14.941579 18225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } attempt: 4
I20260812 06:59:14.942926 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.942346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33632)
0812 06:59:14.942640 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.942759 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 06:59:14.943938 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.943598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33638)
0812 06:59:14.943755 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.943833 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 06:59:14.944166 18225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:40699: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
I20260812 06:59:14.951771 18094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } attempt: 3
I20260812 06:59:14.952980 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.952476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33654)
0812 06:59:14.952686 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.952769 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 06:59:14.954049 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:14.953666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33660)
0812 06:59:14.953882 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:14.953949 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260812 06:59:14.954242 18094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:40699: Network error: Client connection negotiation failed: client connection to 127.26.226.60:40699: connect: Connection refused (error 111)
W20260812 06:59:14.963930 18236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:14.964344 18236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:14.964515 18236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:14.971858 18236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:14.972004 18236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:14.972136 18236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:14.972245 18236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:14.972330 18236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:14.982537 18236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:34613,127.26.226.61:46691,127.26.226.60:40699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab
--principal=kudu/127.26.226.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.60:40699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.226.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:14.983942 18236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:14.985265 18236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:14.997460 18246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:14.997504 18250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:14.998257 18245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:14.998558 18236 server_base.cc:1034] running on GCE node
I20260812 06:59:14.999521 18236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:15.001055 18236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:15.002313 18236 hybrid_clock.cc:648] HybridClock initialized: now 1786517955002253 us; error 50 us; skew 500 ppm
I20260812 06:59:15.002699 18236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:15 dist-test-slave-t3q3 krb5kdc[17951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:15.006018 18236 init.cc:377] Logged in from keytab as kudu/127.26.226.60@KRBTEST.COM (short username kudu)
I20260812 06:59:15.007591 18236 webserver.cc:460] Webserver started at http://127.26.226.60:44643/ using document root <none> and password file <none>
I20260812 06:59:15.008383 18236 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:15.008538 18236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:15.008853 18236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:15.011917 18236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/data/instance:
uuid: "f49a964c5ccd4c96824c38723522cf94"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-t3q3"
server_key: "e62d66b91974ee48c0b5cf8122344738"
server_key_iv: "a6c63b1d77a3745908c81e375a860ab3"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.012825 18236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/wal/instance:
uuid: "f49a964c5ccd4c96824c38723522cf94"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-t3q3"
server_key: "e62d66b91974ee48c0b5cf8122344738"
server_key_iv: "a6c63b1d77a3745908c81e375a860ab3"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.018131 18236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:59:15.021739 18264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:15.023124 18236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:59:15.023327 18236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/wal
uuid: "f49a964c5ccd4c96824c38723522cf94"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-t3q3"
server_key: "e62d66b91974ee48c0b5cf8122344738"
server_key_iv: "a6c63b1d77a3745908c81e375a860ab3"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.023538 18236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:15.043865 18236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:15.048715 18236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:15.048991 18236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:15.067759 18236 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.60:40699
I20260812 06:59:15.067796 18345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.60:40699 every 8 connection(s)
I20260812 06:59:15.069231 18236 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
I20260812 06:59:15.070899 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 18236
I20260812 06:59:15.071122 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517734245872-27528-0/minicluster-data/master-2/wal/instance
I20260812 06:59:15.071663 27528 external_mini_cluster.cc:1428] Setting key cc074c93335ec462ea9fe5ab081e6d12
I20260812 06:59:15.073709 18346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:15.083148 18346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:59:15 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
Aug 12 06:59:15 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:15.097256 18349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.073676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.26.226.60:40699)
0812 06:59:15.074587 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.074606 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.074656 (+ 50us) server_negotiation.cc:407] Connection header received
0812 06:59:15.076835 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.076862 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.077386 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.077824 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.079256 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.081321 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.082800 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.083886 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.084119 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.088898 (+ 4779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.088909 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.088926 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.088975 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.092561 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.093581 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.093588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.093595 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.093832 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.094278 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.094284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.094289 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.094960 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.095274 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.095741 (+ 467us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.096197 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":375,"threads_started":1}
I20260812 06:59:15.102350 18350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.086072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47518 (local address 127.26.226.62:34613)
0812 06:59:15.086636 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.086642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.087216 (+ 574us) server_negotiation.cc:407] Connection header received
0812 06:59:15.087429 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.087438 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.087536 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.087654 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.089020 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.089965 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.092122 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.092548 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.092699 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.096331 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.096336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.096340 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.096375 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.099169 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.100142 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.100147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.100150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.100324 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.100818 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.100822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.100825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.101190 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.101328 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.101886 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.102046 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":279,"threads_started":1}
I20260812 06:59:15.102627 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.086032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34613 (local address 127.0.0.1:47518)
0812 06:59:15.087047 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:15.087103 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:15.087280 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:15.087756 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:15.087766 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:15.088490 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:15.088866 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.088877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.090133 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.090139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.091906 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.091922 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.092750 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.092755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.092878 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.093537 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:15.093563 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:15.095896 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:15.099330 (+ 3434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.099343 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.099350 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.099941 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.100456 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.100464 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.100469 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.100670 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.101344 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:15.101351 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:15.101583 (+ 232us) client_negotiation.cc:769] Sending connection context
0812 06:59:15.101807 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 06:59:15.102168 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":629,"threads_started":1}
I20260812 06:59:15.104004 18346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:59:15 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
Aug 12 06:59:15 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 06:59:15.117800 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.100293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.26.226.62:34613)
0812 06:59:15.100997 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.101003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.101027 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:59:15.101070 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.101074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.101141 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.101257 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.102981 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.104034 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.105780 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.106430 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.106529 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.111396 (+ 4867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.111401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.111405 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.111437 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.114865 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.115582 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.115588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.115592 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.115725 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.116194 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.116199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.116204 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.116492 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.116794 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.117110 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.117365 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":319,"threads_started":1}
I20260812 06:59:15.118194 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.104859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:46691 (local address 127.0.0.1:34690)
0812 06:59:15.105019 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:15.105038 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:15.105163 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:15.105501 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:15.105505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:15.105926 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:15.106149 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.106161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.107453 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.107457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.108378 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.108390 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.109391 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.109395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.109548 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.110123 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:15.110148 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:15.112331 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:15.115783 (+ 3452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.115790 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.115799 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.116291 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.116702 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.116708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.116717 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.116853 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.117368 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:15.117373 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:15.117576 (+ 203us) client_negotiation.cc:769] Sending connection context
0812 06:59:15.117733 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 06:59:15.117962 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 06:59:15.118498 18230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.104870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.26.226.61:46691)
0812 06:59:15.105061 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.105067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.105135 (+ 68us) server_negotiation.cc:407] Connection header received
0812 06:59:15.105271 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.105278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.105362 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.105468 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.106305 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.107288 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.108524 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.109235 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.109375 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.112532 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.112541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.112549 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.112600 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.115626 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.116425 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.116432 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.116440 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.116581 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.116983 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.116987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.116989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.117238 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.117404 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.117896 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.118127 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 06:59:15.119606 18346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:59:15 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
Aug 12 06:59:15 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 06:59:15.131460 18094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } attempt: 4
I20260812 06:59:15.135975 18349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.120755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33678 (local address 127.26.226.60:40699)
0812 06:59:15.120915 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.120919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.120933 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:59:15.120969 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.120972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.121020 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.121094 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.122041 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.123143 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.124971 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.125647 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.125781 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.129604 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.129609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.129612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.129644 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.133012 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.133797 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.133804 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.133807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.133955 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.134364 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.134372 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.134376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.134807 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.134960 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.135302 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.135549 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:59:15.136092 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.120518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33678)
0812 06:59:15.120727 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:15.120747 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:15.120847 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:15.121160 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:15.121166 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:15.121688 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:15.121887 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.121899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.123310 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.123318 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.124794 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.124811 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.125835 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.125843 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.126083 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.126782 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:15.126810 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:15.129419 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:15.133161 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.133168 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.133173 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.133701 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.134110 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.134113 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.134115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.134240 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.134921 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:15.134926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:15.135074 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 06:59:15.135223 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 06:59:15.135705 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":172}
I20260812 06:59:15.139819 18230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.119854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.26.226.61:46691)
0812 06:59:15.120019 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.120025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.120168 (+ 143us) server_negotiation.cc:407] Connection header received
0812 06:59:15.120325 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.120331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.120413 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.120518 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.122171 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.123221 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.125101 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.125768 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.125881 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.130925 (+ 5044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.130930 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.130933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.130964 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.135894 (+ 4930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.136827 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.136835 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.136841 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.137010 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.137455 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.137461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.137464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.137894 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.138180 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.139212 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.139451 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 06:59:15 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517953, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
I20260812 06:59:15.142241 18346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94: Bootstrap starting.
I20260812 06:59:15.144986 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:15.146695 18346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:15.148162 18346 log.cc:824] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:15.149574 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.132191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33684)
0812 06:59:15.132413 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:15.132438 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:15.132599 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:15.133312 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:15.133318 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:15.133867 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:15.134099 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.134112 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.135895 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.135901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.137396 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.137412 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.138337 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.138346 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.138536 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.139154 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:15.139179 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:15.141921 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:15.146168 (+ 4247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.146174 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.146179 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.146714 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.147244 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.147249 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.147253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.147440 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.148301 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:15.148306 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:15.148486 (+ 180us) client_negotiation.cc:769] Sending connection context
0812 06:59:15.148661 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 06:59:15.149055 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 06:59:15.150800 18357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.132229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33684 (local address 127.26.226.60:40699)
0812 06:59:15.132941 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.132947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.132984 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:59:15.133031 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.133036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.133111 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.133266 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.134278 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.135718 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.137644 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.138188 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.138321 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.142194 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.142205 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.142211 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.142257 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.146017 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.146884 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.146889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.146893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.147083 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.147590 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.147596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.147600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.148097 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.149986 (+ 1889us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.150128 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.150357 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":316,"threads_started":1}
I20260812 06:59:15.152108 18346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94: No bootstrap required, opened a new log
I20260812 06:59:15.157966 18094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4: Bootstrap starting.
I20260812 06:59:15.162380 18094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:15.163816 18094 log.cc:824] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:15.167912 18346 raft_consensus.cc:348] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } }
I20260812 06:59:15.170375 18346 raft_consensus.cc:374] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:15.170567 18346 raft_consensus.cc:729] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f49a964c5ccd4c96824c38723522cf94, State: Initialized, Role: FOLLOWER
I20260812 06:59:15.171375 18346 consensus_queue.cc:260] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } }
I20260812 06:59:15.173636 18357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.153277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33694 (local address 127.26.226.60:40699)
0812 06:59:15.153471 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.153478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.153590 (+ 112us) server_negotiation.cc:407] Connection header received
0812 06:59:15.153901 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.153907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.154011 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.154093 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.156586 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.157724 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.159893 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.160748 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.160877 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.164941 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.164954 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.164960 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.165016 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.169191 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.170159 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.170164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.170167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.170317 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.170765 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.170770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.170773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.171326 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.171617 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.172906 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.173245 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:59:15.176412 18094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4: No bootstrap required, opened a new log
I20260812 06:59:15.183245 18230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.151466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.26.226.61:46691)
0812 06:59:15.151634 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.151638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.152108 (+ 470us) server_negotiation.cc:407] Connection header received
0812 06:59:15.152376 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.152382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.152459 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.152537 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.166320 (+ 13783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.167532 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.169926 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.170616 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.170721 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.175496 (+ 4775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.175503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.175509 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.175555 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.179116 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.180160 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.180165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.180169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.180325 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.180837 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.180843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.180845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.181227 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.181456 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.182600 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.182857 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:59:15.185475 18384 sys_catalog.cc:455] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } } }
I20260812 06:59:15.185927 18384 sys_catalog.cc:458] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:15.187424 18346 sys_catalog.cc:564] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:15.187652 18094 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } }
I20260812 06:59:15.188119 18094 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:15.188293 18094 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2562adff7864bad866590ca850e5bc4, State: Initialized, Role: FOLLOWER
I20260812 06:59:15.188949 18094 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } }
I20260812 06:59:15.189659 18384 raft_consensus.cc:482] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:15.190069 18384 raft_consensus.cc:504] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } }
I20260812 06:59:15.190893 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.153464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47528 (local address 127.26.226.62:34613)
0812 06:59:15.153681 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.153689 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.169931 (+ 16242us) server_negotiation.cc:407] Connection header received
0812 06:59:15.170254 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.170260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.170333 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.170414 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.172506 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.173869 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.176509 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.177455 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.177581 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.182291 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.182300 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.182308 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.182366 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.186937 (+ 4571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.187809 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.187814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.187817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.188001 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.188527 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.188532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.188536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.188965 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.189301 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.190038 (+ 737us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.190368 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 06:59:15.194862 18384 leader_election.cc:288] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2562adff7864bad866590ca850e5bc4 (127.26.226.62:34613), d391094ea5ef4cb8acd367121eac200d (127.26.226.61:46691)
I20260812 06:59:15.197218 18094 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:15.197762 18050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f49a964c5ccd4c96824c38723522cf94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2562adff7864bad866590ca850e5bc4" is_pre_election: true
I20260812 06:59:15.198560 18050 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f49a964c5ccd4c96824c38723522cf94 in term 0.
I20260812 06:59:15.199568 18274 leader_election.cc:302] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e2562adff7864bad866590ca850e5bc4, f49a964c5ccd4c96824c38723522cf94; no voters:
I20260812 06:59:15.200096 18384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:15.200290 18384 raft_consensus.cc:482] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:15.200410 18384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:15.200731 18388 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } } }
I20260812 06:59:15.201256 18388 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:15.205076 18384 raft_consensus.cc:504] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } }
I20260812 06:59:15.206468 18384 leader_election.cc:288] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [CANDIDATE]: Term 1 election: Requested vote from peers e2562adff7864bad866590ca850e5bc4 (127.26.226.62:34613), d391094ea5ef4cb8acd367121eac200d (127.26.226.61:46691)
I20260812 06:59:15.206517 18050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f49a964c5ccd4c96824c38723522cf94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2562adff7864bad866590ca850e5bc4"
I20260812 06:59:15.207171 18050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:15.211197 18050 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f49a964c5ccd4c96824c38723522cf94 in term 1.
I20260812 06:59:15.211746 18274 leader_election.cc:302] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e2562adff7864bad866590ca850e5bc4, f49a964c5ccd4c96824c38723522cf94; no voters:
I20260812 06:59:15.212155 18384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:15.212927 18384 raft_consensus.cc:686] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [term 1 LEADER]: Becoming Leader. State: Replica: f49a964c5ccd4c96824c38723522cf94, State: Running, Role: LEADER
I20260812 06:59:15.213766 18384 consensus_queue.cc:237] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } }
I20260812 06:59:15.225517 18407 sys_catalog.cc:455] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f49a964c5ccd4c96824c38723522cf94. Latest consensus state: current_term: 1 leader_uuid: "f49a964c5ccd4c96824c38723522cf94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } } }
I20260812 06:59:15.226156 18407 sys_catalog.cc:458] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:15.227105 18424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260812 06:59:15.230255 18017 tablet.cc:2369] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:15.242241 18424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:15.243364 18225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } attempt: 5
Aug 12 06:59:15 dist-test-slave-t3q3 krb5kdc[17951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517954, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
I20260812 06:59:15.262449 18431 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 06:59:15.267795 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.244055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:40699 (local address 127.0.0.1:33702)
0812 06:59:15.244564 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:15.244591 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:15.244733 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:15.245684 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:15.245690 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:15.246262 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:15.246486 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.246504 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.248147 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.248152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.249579 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.249599 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.251266 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.251272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.251525 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.252374 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:15.252400 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:15.255464 (+ 3064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:15.263782 (+ 8318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.263790 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.263795 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.264423 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.265250 (+ 827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.265255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.265259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.265431 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.266698 (+ 1267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:15.266703 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:15.266875 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 06:59:15.267041 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 06:59:15.267358 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295}
I20260812 06:59:15.270078 18357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.245019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33702 (local address 127.26.226.60:40699)
0812 06:59:15.245178 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.245185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.245203 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:59:15.245264 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.245268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.245355 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.245465 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.246685 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.247916 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.250141 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.250878 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.251016 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.255749 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.255756 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.255761 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.255803 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.263524 (+ 7721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.264616 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.264659 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.264666 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.265033 (+ 367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.265580 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.265585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.265622 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.266495 (+ 873us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.266924 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.267373 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.267921 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 06:59:15.270432 18050 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [term 1 FOLLOWER]: Refusing update from remote peer f49a964c5ccd4c96824c38723522cf94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:15.271492 18407 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:15.277456 18225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d: Bootstrap starting.
I20260812 06:59:15.264955 18236 master_runner.cc:186] Error getting master registration for 127.26.226.62:34613: OK, instance_id { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" instance_seqno: 1786517953990406 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:59:15.277940 18236 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:59:15.284713 18225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:15.287173 18225 log.cc:824] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:15.296093 18388 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f49a964c5ccd4c96824c38723522cf94. Latest consensus state: current_term: 1 leader_uuid: "f49a964c5ccd4c96824c38723522cf94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } } }
I20260812 06:59:15.296530 18388 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 06:59:15.301142 18277 tablet.cc:2369] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:15.302217 18433 mvcc.cc:205] Tried to move back new op lower bound from 7317577544741306368 to 7317577544561762304. Current Snapshot: MvccSnapshot[applied={T|T < 7317577544741306368}]
I20260812 06:59:15.302968 18225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d: No bootstrap required, opened a new log
I20260812 06:59:15.304462 18437 mvcc.cc:205] Tried to move back new op lower bound from 7317577544741306368 to 7317577544561762304. Current Snapshot: MvccSnapshot[applied={T|T < 7317577544741306368}]
I20260812 06:59:15.313901 18388 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f49a964c5ccd4c96824c38723522cf94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } } }
I20260812 06:59:15.314225 18388 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:15.318075 18384 sys_catalog.cc:455] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f49a964c5ccd4c96824c38723522cf94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } } }
I20260812 06:59:15.318634 18384 sys_catalog.cc:458] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:15.327602 18225 raft_consensus.cc:348] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } }
I20260812 06:59:15.328105 18225 raft_consensus.cc:374] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:15.328251 18225 raft_consensus.cc:729] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d391094ea5ef4cb8acd367121eac200d, State: Initialized, Role: FOLLOWER
I20260812 06:59:15.328955 18424 catalog_manager.cc:1347] Generated new cluster ID: 31ba4cbf339149e9894c5375ac9e9dfd
I20260812 06:59:15.329059 18424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:15.328944 18225 consensus_queue.cc:260] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } }
I20260812 06:59:15.336931 18446 sys_catalog.cc:455] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } } }
I20260812 06:59:15.337359 18446 sys_catalog.cc:458] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:15.339304 18225 sys_catalog.cc:564] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:15.339380 18184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f49a964c5ccd4c96824c38723522cf94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d391094ea5ef4cb8acd367121eac200d"
I20260812 06:59:15.339803 18183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:15.339329 18185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f49a964c5ccd4c96824c38723522cf94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d391094ea5ef4cb8acd367121eac200d" is_pre_election: true
I20260812 06:59:15.343860 18183 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [term 1 FOLLOWER]: Refusing update from remote peer f49a964c5ccd4c96824c38723522cf94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:15.344890 18407 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:15.366042 18424 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:15.368474 18424 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:15.353724 17973 master_runner.cc:186] Error getting master registration for 127.26.226.61:46691: OK, instance_id { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" instance_seqno: 1786517954511911 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:59:15.379722 17973 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:59:15.384598 18441 catalog_manager.cc:1259] Loaded cluster ID: 31ba4cbf339149e9894c5375ac9e9dfd
I20260812 06:59:15.384774 18441 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4: loading cluster ID for follower catalog manager: success
I20260812 06:59:15.388427 18441 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e2562adff7864bad866590ca850e5bc4: acquiring CA information for follower catalog manager: success
W20260812 06:59:15.389825 18441 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260812 06:59:15.402458 18424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94: Generated new TSK 0
I20260812 06:59:15.403817 18424 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:15.429162 18407 sys_catalog.cc:455] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f49a964c5ccd4c96824c38723522cf94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } } }
I20260812 06:59:15.429493 18407 sys_catalog.cc:458] T 00000000000000000000000000000000 P f49a964c5ccd4c96824c38723522cf94 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:15.432986 18446 sys_catalog.cc:455] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f49a964c5ccd4c96824c38723522cf94. Latest consensus state: current_term: 1 leader_uuid: "f49a964c5ccd4c96824c38723522cf94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } } }
I20260812 06:59:15.433388 18446 sys_catalog.cc:458] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:15.432996 18462 sys_catalog.cc:455] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f49a964c5ccd4c96824c38723522cf94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2562adff7864bad866590ca850e5bc4" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34613 } } peers { permanent_uuid: "d391094ea5ef4cb8acd367121eac200d" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 46691 } } peers { permanent_uuid: "f49a964c5ccd4c96824c38723522cf94" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 40699 } } }
I20260812 06:59:15.433983 18462 sys_catalog.cc:458] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:15.503257 18481 catalog_manager.cc:1259] Loaded cluster ID: 31ba4cbf339149e9894c5375ac9e9dfd
I20260812 06:59:15.503381 18481 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d: loading cluster ID for follower catalog manager: success
I20260812 06:59:15.507771 18481 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d: acquiring CA information for follower catalog manager: success
I20260812 06:59:15.511646 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 17973
I20260812 06:59:15.513077 18085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:15.522301 18481 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d391094ea5ef4cb8acd367121eac200d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:59:15.646608 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 17973
I20260812 06:59:15.677800 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 18106
I20260812 06:59:15.679030 18218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:15.782842 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 18106
I20260812 06:59:15.808267 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 18236
I20260812 06:59:15.809408 18336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:15.933265 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 18236
2026-08-12T06:59:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4112 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:16 dist-test-slave-t3q3 krb5kdc[18491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:16 dist-test-slave-t3q3 krb5kdc[18491](info): set up 2 sockets
Aug 12 06:59:16 dist-test-slave-t3q3 krb5kdc[18491](info): commencing operation
krb5kdc: starting...
W20260812 06:59:17.374850 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:59:17 dist-test-slave-t3q3 krb5kdc[18491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:59:17.695659 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:32985
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:33437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:32985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:18.055055 18514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:18.055365 18514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:18.055569 18514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:18.062914 18514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:18.063031 18514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:18.063151 18514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:18.063261 18514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:18.063347 18514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:18.073546 18514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:33437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:32985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:32985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:18.074928 18514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:18.076319 18514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:18.088562 18526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:18.088937 18533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:18.088980 18524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:18.089632 18514 server_base.cc:1034] running on GCE node
I20260812 06:59:18.090529 18514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:18.092027 18514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:18.093309 18514 hybrid_clock.cc:648] HybridClock initialized: now 1786517958093252 us; error 55 us; skew 500 ppm
I20260812 06:59:18.093699 18514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:18 dist-test-slave-t3q3 krb5kdc[18491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517958, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:18.097115 18514 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:59:18.098685 18514 webserver.cc:460] Webserver started at http://127.26.226.62:44183/ using document root <none> and password file <none>
I20260812 06:59:18.099391 18514 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:18.099547 18514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:18.099885 18514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:18.103084 18514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "1dda519c461b45b09853706050fe6423"
format_stamp: "Formatted at 2026-08-12 06:59:18 on dist-test-slave-t3q3"
server_key: "a7556540953b88a50bd3c0bcc9343957"
server_key_iv: "3f86bf87dd49dc51393f64119fb5ed33"
server_key_version: "encryptionkey@0"
I20260812 06:59:18.103875 18514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "1dda519c461b45b09853706050fe6423"
format_stamp: "Formatted at 2026-08-12 06:59:18 on dist-test-slave-t3q3"
server_key: "a7556540953b88a50bd3c0bcc9343957"
server_key_iv: "3f86bf87dd49dc51393f64119fb5ed33"
server_key_version: "encryptionkey@0"
I20260812 06:59:18.109457 18514 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 06:59:18.113049 18545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:18.114377 18514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 06:59:18.114588 18514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "1dda519c461b45b09853706050fe6423"
format_stamp: "Formatted at 2026-08-12 06:59:18 on dist-test-slave-t3q3"
server_key: "a7556540953b88a50bd3c0bcc9343957"
server_key_iv: "3f86bf87dd49dc51393f64119fb5ed33"
server_key_version: "encryptionkey@0"
I20260812 06:59:18.114810 18514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:18.131414 18514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:18.135629 18514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:18.135944 18514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:18.155730 18514 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:32985
I20260812 06:59:18.155761 18631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:32985 every 8 connection(s)
I20260812 06:59:18.157228 18514 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:18.161084 18634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:18.167111 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 18514
I20260812 06:59:18.167367 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:59:18.168320 27528 external_mini_cluster.cc:1428] Setting key 8d7f4f6abf11a28f21f9ea96e31e137d
I20260812 06:59:18.173012 18634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423: Bootstrap starting.
I20260812 06:59:18.176239 18634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:18.177333 18634 log.cc:824] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:18.180258 18634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423: No bootstrap required, opened a new log
Aug 12 06:59:18 dist-test-slave-t3q3 krb5kdc[18491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:18.188828 18634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dda519c461b45b09853706050fe6423" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 32985 } }
I20260812 06:59:18.189254 18634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:18.189389 18634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dda519c461b45b09853706050fe6423, State: Initialized, Role: FOLLOWER
I20260812 06:59:18.190028 18634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dda519c461b45b09853706050fe6423" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 32985 } }
I20260812 06:59:18.190325 18634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:18.190496 18634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:18.190716 18634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:18.192461 18640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:18.170449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56546 (local address 127.26.226.62:32985)
0812 06:59:18.171291 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:18.171303 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:18.171346 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:59:18.173224 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:18.173249 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:18.173636 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:18.174014 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:18.175572 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:18.177008 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:18.178845 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:18.179517 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:18.179722 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:18.184386 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:18.184394 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:18.184414 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:18.184466 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:18.187799 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:18.188656 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:18.188663 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:18.188670 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:18.188844 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:18.189317 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:18.189322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:18.189327 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:18.190068 (+ 741us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:18.190400 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:18.190837 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 06:59:18.191303 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":395,"threads_started":1}
I20260812 06:59:18.194110 18634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dda519c461b45b09853706050fe6423" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 32985 } }
I20260812 06:59:18.194643 18634 leader_election.cc:302] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dda519c461b45b09853706050fe6423; no voters:
I20260812 06:59:18.195336 18634 leader_election.cc:288] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:18.195502 18643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:18.197299 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:18.197265 18643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [term 1 LEADER]: Becoming Leader. State: Replica: 1dda519c461b45b09853706050fe6423, State: Running, Role: LEADER
I20260812 06:59:18.197757 18634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:18.197814 18643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dda519c461b45b09853706050fe6423" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 32985 } }
I20260812 06:59:18.200654 18645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dda519c461b45b09853706050fe6423. Latest consensus state: current_term: 1 leader_uuid: "1dda519c461b45b09853706050fe6423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dda519c461b45b09853706050fe6423" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 32985 } } }
I20260812 06:59:18.201086 18645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:18.200964 18644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dda519c461b45b09853706050fe6423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dda519c461b45b09853706050fe6423" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 32985 } } }
I20260812 06:59:18.201313 18644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:18.204917 18653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:18.210368 18653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:18.219977 18653 catalog_manager.cc:1347] Generated new cluster ID: 5f66faa92bbc40e3b88ee442a4213cdb
I20260812 06:59:18.220155 18653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:18.239837 18653 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:18.241297 18653 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:18.254065 18653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dda519c461b45b09853706050fe6423: Generated new TSK 0
I20260812 06:59:18.255038 18653 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:18.312193 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 18514
I20260812 06:59:18.313572 18624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:18.414848 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 18514
2026-08-12T06:59:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2473 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:18 dist-test-slave-t3q3 krb5kdc[18681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:18 dist-test-slave-t3q3 krb5kdc[18681](info): set up 2 sockets
Aug 12 06:59:18 dist-test-slave-t3q3 krb5kdc[18681](info): commencing operation
krb5kdc: starting...
W20260812 06:59:19.784832 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.006s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:59:19 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:59:20.103317 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34519
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:34519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:20.462921 18707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:20.463308 18707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:20.463476 18707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:20.470965 18707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:20.471150 18707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:20.471268 18707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:20.471380 18707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:20.471472 18707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:20.481976 18707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:34519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:20.483507 18707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:20.484896 18707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:20.496837 18721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:20.496909 18718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:20.496933 18717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:20.498049 18707 server_base.cc:1034] running on GCE node
I20260812 06:59:20.498940 18707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:20.500432 18707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:20.501690 18707 hybrid_clock.cc:648] HybridClock initialized: now 1786517960501619 us; error 52 us; skew 500 ppm
I20260812 06:59:20.502107 18707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:20 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517960, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:20.510434 18707 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:59:20.512190 18707 webserver.cc:460] Webserver started at http://127.26.226.62:43207/ using document root <none> and password file <none>
I20260812 06:59:20.512928 18707 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:20.513101 18707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:20.513417 18707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:20.516536 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "42d525d1eb154a16ae71ced739fea360"
format_stamp: "Formatted at 2026-08-12 06:59:20 on dist-test-slave-t3q3"
server_key: "8b7a27faa52ebc95fc205c5596058ac5"
server_key_iv: "b72264e01947e8c04a91847900f05367"
server_key_version: "encryptionkey@0"
I20260812 06:59:20.517274 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "42d525d1eb154a16ae71ced739fea360"
format_stamp: "Formatted at 2026-08-12 06:59:20 on dist-test-slave-t3q3"
server_key: "8b7a27faa52ebc95fc205c5596058ac5"
server_key_iv: "b72264e01947e8c04a91847900f05367"
server_key_version: "encryptionkey@0"
I20260812 06:59:20.522744 18707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:59:20.526356 18731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:20.527834 18707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 06:59:20.528029 18707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "42d525d1eb154a16ae71ced739fea360"
format_stamp: "Formatted at 2026-08-12 06:59:20 on dist-test-slave-t3q3"
server_key: "8b7a27faa52ebc95fc205c5596058ac5"
server_key_iv: "b72264e01947e8c04a91847900f05367"
server_key_version: "encryptionkey@0"
I20260812 06:59:20.528298 18707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:20.547590 18707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:20.552428 18707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:20.552820 18707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:20.573566 18707 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:34519
I20260812 06:59:20.573716 18822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:34519 every 8 connection(s)
I20260812 06:59:20.575232 18707 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:20.579149 18823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:20.584920 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 18707
I20260812 06:59:20.585188 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:59:20.585883 27528 external_mini_cluster.cc:1428] Setting key a1500dd08f0496bfd60a767fbc2fa0ef
I20260812 06:59:20.591477 18823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Bootstrap starting.
I20260812 06:59:20.595517 18823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:20.596844 18823 log.cc:824] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:20.600459 18823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: No bootstrap required, opened a new log
Aug 12 06:59:20 dist-test-slave-t3q3 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:20.611229 18823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:20.611824 18823 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:20.611946 18823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42d525d1eb154a16ae71ced739fea360, State: Initialized, Role: FOLLOWER
I20260812 06:59:20.612725 18823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:20.613070 18823 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:20.613232 18823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:20.613425 18823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:20.613801 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:20.588234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.26.226.62:34519)
0812 06:59:20.589095 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:20.589111 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:20.589175 (+ 64us) server_negotiation.cc:407] Connection header received
0812 06:59:20.591657 (+ 2482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:20.591689 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:20.592256 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:20.592716 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:20.594191 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:20.596236 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:20.597919 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:20.598895 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:20.599097 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:20.603717 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:20.603730 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:20.603758 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:20.603818 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:20.608437 (+ 4619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:20.609316 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:20.609331 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:20.609342 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:20.609584 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:20.610085 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:20.610095 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:20.610099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:20.610913 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:20.611251 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:20.611683 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 06:59:20.612234 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":357,"threads_started":1}
I20260812 06:59:20.616166 18823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:20.616715 18823 leader_election.cc:302] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42d525d1eb154a16ae71ced739fea360; no voters:
I20260812 06:59:20.617436 18823 leader_election.cc:288] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:20.617544 18831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:20.618242 18831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 1 LEADER]: Becoming Leader. State: Replica: 42d525d1eb154a16ae71ced739fea360, State: Running, Role: LEADER
I20260812 06:59:20.618861 18831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:20.619872 18823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:20.622337 18832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42d525d1eb154a16ae71ced739fea360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } } }
I20260812 06:59:20.622732 18832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:20.624782 18833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42d525d1eb154a16ae71ced739fea360. Latest consensus state: current_term: 1 leader_uuid: "42d525d1eb154a16ae71ced739fea360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } } }
I20260812 06:59:20.625036 18841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:20.625156 18833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:20.632015 18841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:20.643352 18841 catalog_manager.cc:1347] Generated new cluster ID: 7c1a1b584a6447ca933aaab8c40e3ea3
I20260812 06:59:20.643507 18841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:20.674443 18841 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:20.676100 18841 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:20.684825 18841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Generated new TSK 0
I20260812 06:59:20.685767 18841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:59:20.781900 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34519
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:21.141287 18870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:21.141577 18870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:21.141769 18870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:21.149107 18870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:21.149231 18870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:21.149437 18870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:59:21.159505 18870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:21.160925 18870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:21.162281 18870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:21.175329 18886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:21.175410 18890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:21.175675 18885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:21.176311 18870 server_base.cc:1034] running on GCE node
I20260812 06:59:21.177034 18870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:21.178054 18870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:21.179311 18870 hybrid_clock.cc:648] HybridClock initialized: now 1786517961179262 us; error 41 us; skew 500 ppm
I20260812 06:59:21.179669 18870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:21 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517961, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:21.183183 18870 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:59:21.185009 18870 webserver.cc:460] Webserver started at http://127.26.226.1:38529/ using document root <none> and password file <none>
I20260812 06:59:21.185755 18870 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:21.185911 18870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:21.186236 18870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:21.189451 18870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "a89275d8057949eb9c552e054bd53c79"
format_stamp: "Formatted at 2026-08-12 06:59:21 on dist-test-slave-t3q3"
server_key: "44e3ea896e40631072f3ee570f6a77b7"
server_key_iv: "c23ce8e53efc425f4f8928ac74c01921"
server_key_version: "encryptionkey@0"
I20260812 06:59:21.190304 18870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "a89275d8057949eb9c552e054bd53c79"
format_stamp: "Formatted at 2026-08-12 06:59:21 on dist-test-slave-t3q3"
server_key: "44e3ea896e40631072f3ee570f6a77b7"
server_key_iv: "c23ce8e53efc425f4f8928ac74c01921"
server_key_version: "encryptionkey@0"
I20260812 06:59:21.195940 18870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:59:21.199780 18899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:21.201274 18870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:21.201478 18870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "a89275d8057949eb9c552e054bd53c79"
format_stamp: "Formatted at 2026-08-12 06:59:21 on dist-test-slave-t3q3"
server_key: "44e3ea896e40631072f3ee570f6a77b7"
server_key_iv: "c23ce8e53efc425f4f8928ac74c01921"
server_key_version: "encryptionkey@0"
I20260812 06:59:21.201722 18870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:21.228777 18870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:21.233930 18870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:21.234261 18870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:21.235329 18870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:21.236876 18870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:21.236953 18870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:21.237114 18870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:21.237182 18870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:21.270715 18870 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:43477
I20260812 06:59:21.270766 19100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:43477 every 8 connection(s)
I20260812 06:59:21.272167 18870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:21.274189 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 18870
I20260812 06:59:21.274430 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:59:21.275365 27528 external_mini_cluster.cc:1428] Setting key 6ec9c0a3446a493a58d9c47d25405d9d
Aug 12 06:59:21 dist-test-slave-t3q3 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517961, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:21.294850 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:21.275077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:53681 (local address 127.26.226.62:34519)
0812 06:59:21.275638 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:21.275644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:21.276955 (+ 1311us) server_negotiation.cc:407] Connection header received
0812 06:59:21.278926 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:21.278932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:21.279023 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:21.279151 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:21.280774 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:21.281519 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:21.283160 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:21.283579 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:21.283736 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:21.287533 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:21.287538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:21.287541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:21.287576 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:21.291082 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:21.292257 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:21.292261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:21.292264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:21.292401 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:21.292916 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:21.292920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:21.292922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:21.293252 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:21.293423 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:21.294219 (+ 796us) server_negotiation.cc:299] Negotiation successful
0812 06:59:21.294464 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":313,"threads_started":1}
I20260812 06:59:21.295900 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:21.275753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:53681)
0812 06:59:21.276770 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:21.276837 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:21.278642 (+ 1805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:21.279411 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:21.279419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:21.279939 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:21.280598 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:21.280616 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:21.281696 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:21.281706 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:21.282988 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:21.283003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:21.283840 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:21.283849 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:21.284184 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:21.284945 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:21.284970 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:21.287278 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:21.291314 (+ 4036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:21.291328 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:21.291350 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:21.292058 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:21.292549 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:21.292555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:21.292558 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:21.292775 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:21.293404 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:21.293413 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:21.293799 (+ 386us) client_negotiation.cc:769] Sending connection context
0812 06:59:21.294085 (+ 286us) client_negotiation.cc:240] Negotiation successful
0812 06:59:21.294681 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":454,"threads_started":1}
I20260812 06:59:21.297767 19101 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34519
I20260812 06:59:21.298117 19101 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:21.298910 19101 heartbeater.cc:511] Master 127.26.226.62:34519 requested a full tablet report, sending...
I20260812 06:59:21.301561 18756 ts_manager.cc:194] Registered new tserver with Master: a89275d8057949eb9c552e054bd53c79 (127.26.226.1:43477)
I20260812 06:59:21.303871 18756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:53681
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:59:21.348052 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34519
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:21.702008 19115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:21.702284 19115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:21.702452 19115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:21.709975 19115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:21.710086 19115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:21.710301 19115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:59:21.720028 19115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:21.721397 19115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:21.722797 19115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:21.736335 19133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:21.736434 19127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:21.736418 19126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:21.737440 19115 server_base.cc:1034] running on GCE node
I20260812 06:59:21.738132 19115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:21.739166 19115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:21.740430 19115 hybrid_clock.cc:648] HybridClock initialized: now 1786517961740364 us; error 51 us; skew 500 ppm
I20260812 06:59:21.740782 19115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:21 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517961, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:21.744356 19115 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:59:21.746047 19115 webserver.cc:460] Webserver started at http://127.26.226.2:39435/ using document root <none> and password file <none>
I20260812 06:59:21.746773 19115 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:21.746932 19115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:21.747260 19115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:21.750439 19115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "5983c4830d8e4b91af2dde459d243f21"
format_stamp: "Formatted at 2026-08-12 06:59:21 on dist-test-slave-t3q3"
server_key: "bc2bc8287b54a2917ae279d4e5623b4e"
server_key_iv: "a30af246a7d1df448779c6129f9daa04"
server_key_version: "encryptionkey@0"
I20260812 06:59:21.751269 19115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "5983c4830d8e4b91af2dde459d243f21"
format_stamp: "Formatted at 2026-08-12 06:59:21 on dist-test-slave-t3q3"
server_key: "bc2bc8287b54a2917ae279d4e5623b4e"
server_key_iv: "a30af246a7d1df448779c6129f9daa04"
server_key_version: "encryptionkey@0"
I20260812 06:59:21.756711 19115 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 06:59:21.760458 19144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:21.761914 19115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:59:21.762066 19115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "5983c4830d8e4b91af2dde459d243f21"
format_stamp: "Formatted at 2026-08-12 06:59:21 on dist-test-slave-t3q3"
server_key: "bc2bc8287b54a2917ae279d4e5623b4e"
server_key_iv: "a30af246a7d1df448779c6129f9daa04"
server_key_version: "encryptionkey@0"
I20260812 06:59:21.762249 19115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:21.801923 19115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:21.806679 19115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:21.807013 19115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:21.808307 19115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:21.809794 19115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:21.809876 19115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:21.809964 19115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:21.810010 19115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:21.846756 19115 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:40395
I20260812 06:59:21.846853 19354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:40395 every 8 connection(s)
I20260812 06:59:21.848150 19115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:21.851634 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 19115
I20260812 06:59:21.851841 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:59:21.852574 27528 external_mini_cluster.cc:1428] Setting key 9601e202517e88bb50c853fecf481164
Aug 12 06:59:21 dist-test-slave-t3q3 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517961, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:21.874543 19359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:21.850954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:38521 (local address 127.26.226.62:34519)
0812 06:59:21.851463 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:21.851471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:21.853301 (+ 1830us) server_negotiation.cc:407] Connection header received
0812 06:59:21.855154 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:21.855158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:21.855225 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:21.855314 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:21.857639 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:21.858397 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:21.861210 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:21.861874 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:21.862005 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:21.867114 (+ 5109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:21.867123 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:21.867131 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:21.867203 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:21.870442 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:21.871607 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:21.871612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:21.871614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:21.871771 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:21.872407 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:21.872411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:21.872414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:21.872747 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:21.872878 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:21.874039 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0812 06:59:21.874241 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":220,"threads_started":1}
I20260812 06:59:21.876202 19360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:21.851570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:38521)
0812 06:59:21.852491 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:21.852574 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:21.854820 (+ 2246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:21.855633 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:21.855645 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:21.856500 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:21.857415 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:21.857445 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:21.858580 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:21.858588 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:21.861024 (+ 2436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:21.861043 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:21.862146 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:21.862156 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:21.862515 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:21.863390 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:21.863429 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:21.866718 (+ 3289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:21.870651 (+ 3933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:21.870665 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:21.870688 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:21.871414 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:21.871907 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:21.871915 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:21.871922 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:21.872240 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:21.872905 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:21.872916 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:21.873472 (+ 556us) client_negotiation.cc:769] Sending connection context
0812 06:59:21.873882 (+ 410us) client_negotiation.cc:240] Negotiation successful
0812 06:59:21.874520 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":431,"threads_started":1}
I20260812 06:59:21.878062 19355 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34519
I20260812 06:59:21.878420 19355 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:21.879429 19355 heartbeater.cc:511] Master 127.26.226.62:34519 requested a full tablet report, sending...
I20260812 06:59:21.881352 18756 ts_manager.cc:194] Registered new tserver with Master: 5983c4830d8e4b91af2dde459d243f21 (127.26.226.2:40395)
I20260812 06:59:21.882722 18756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:38521
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:59:21.932178 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34519
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:22.290184 19363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:22.290540 19363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:22.290727 19363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:22.298059 19363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:22.298223 19363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:22.298419 19363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:59:22.307579 19101 heartbeater.cc:503] Master 127.26.226.62:34519 was elected leader, sending a full tablet report...
I20260812 06:59:22.308952 19363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:22.310449 19363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:22.311750 19363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:22.325268 19371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:22.325366 19373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:22.325294 19377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:22.326438 19363 server_base.cc:1034] running on GCE node
I20260812 06:59:22.327092 19363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:22.328192 19363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:22.329449 19363 hybrid_clock.cc:648] HybridClock initialized: now 1786517962329399 us; error 54 us; skew 500 ppm
I20260812 06:59:22.329795 19363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:22 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517962, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:22.333532 19363 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:59:22.335114 19363 webserver.cc:460] Webserver started at http://127.26.226.3:45585/ using document root <none> and password file <none>
I20260812 06:59:22.335848 19363 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:22.336007 19363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:22.336364 19363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:22.339394 19363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7"
format_stamp: "Formatted at 2026-08-12 06:59:22 on dist-test-slave-t3q3"
server_key: "31f796a6f63903e76dae46e51df1c5be"
server_key_iv: "f0beb68f6450b4a08d16cf8b9b2604cf"
server_key_version: "encryptionkey@0"
I20260812 06:59:22.340286 19363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7"
format_stamp: "Formatted at 2026-08-12 06:59:22 on dist-test-slave-t3q3"
server_key: "31f796a6f63903e76dae46e51df1c5be"
server_key_iv: "f0beb68f6450b4a08d16cf8b9b2604cf"
server_key_version: "encryptionkey@0"
I20260812 06:59:22.345695 19363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 06:59:22.350940 19390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:22.352489 19363 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.002s
I20260812 06:59:22.352632 19363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7"
format_stamp: "Formatted at 2026-08-12 06:59:22 on dist-test-slave-t3q3"
server_key: "31f796a6f63903e76dae46e51df1c5be"
server_key_iv: "f0beb68f6450b4a08d16cf8b9b2604cf"
server_key_version: "encryptionkey@0"
I20260812 06:59:22.352804 19363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:22.403800 19363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:22.408593 19363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:22.408850 19363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:22.409782 19363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:22.411123 19363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:22.411207 19363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:22.411298 19363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:22.411342 19363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:22.445258 19363 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:43509
I20260812 06:59:22.445349 19567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:43509 every 8 connection(s)
I20260812 06:59:22.446570 19363 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:22.457028 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 19363
I20260812 06:59:22.457250 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:59:22.457911 27528 external_mini_cluster.cc:1428] Setting key 1bddbc8cdc1329cd47846ccf37dbef94
Aug 12 06:59:22 dist-test-slave-t3q3 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517962, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:22.466790 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.449163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:33309 (local address 127.26.226.62:34519)
0812 06:59:22.449694 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.449698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.450475 (+ 777us) server_negotiation.cc:407] Connection header received
0812 06:59:22.452409 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.452415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.452526 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.452652 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:22.454247 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.455040 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.456699 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.457228 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.457361 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.461297 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:22.461302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:22.461305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:22.461337 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:22.464013 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:22.464737 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:22.464740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:22.464743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:22.464857 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:22.465258 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:22.465261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:22.465263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:22.465514 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:22.465625 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.466358 (+ 733us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.466521 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":279,"threads_started":1}
I20260812 06:59:22.467806 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.449622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:33309)
0812 06:59:22.450312 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:22.450350 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:22.452113 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:22.452914 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:22.452923 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:22.453455 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:22.454052 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.454071 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.455198 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.455210 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.456541 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.456554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.457482 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.457489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.457727 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.458435 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:22.458462 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:22.461094 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:22.464194 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:22.464201 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:22.464212 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:22.464626 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:22.464970 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:22.464973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:22.464975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:22.465152 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:22.465624 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:22.465629 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:22.466002 (+ 373us) client_negotiation.cc:769] Sending connection context
0812 06:59:22.466252 (+ 250us) client_negotiation.cc:240] Negotiation successful
0812 06:59:22.466698 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":277,"threads_started":1}
I20260812 06:59:22.469314 19568 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34519
I20260812 06:59:22.469656 19568 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:22.470451 19568 heartbeater.cc:511] Master 127.26.226.62:34519 requested a full tablet report, sending...
I20260812 06:59:22.472107 18756 ts_manager.cc:194] Registered new tserver with Master: 2d89bbfec43e4f30bd70dd2d6a6c78c7 (127.26.226.3:43509)
I20260812 06:59:22.473156 18756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:33309
I20260812 06:59:22.473636 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:22.491094 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.478424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40394 (local address 127.26.226.62:34519)
0812 06:59:22.478594 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.478599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.478989 (+ 390us) server_negotiation.cc:407] Connection header received
0812 06:59:22.479201 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.479206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.479286 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.479389 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:22.480870 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.481596 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.482926 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.483352 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.483498 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.485941 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:22.485952 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:22.485956 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:22.485987 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:22.488436 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:22.489114 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:22.489120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:22.489124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:22.489263 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:22.489637 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:22.489642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:22.489645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:22.489964 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:22.490242 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.490554 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.490779 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:59:22.495198 18756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:59:22.502261 18756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:59:22.526054 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.517296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.26.226.1:43477)
0812 06:59:22.518219 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.518226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.518252 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:59:22.518349 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.518358 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.518587 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.519400 (+ 813us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:22.519624 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.521214 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.523179 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.524801 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.524951 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.525207 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.525490 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.525679 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":580,"threads_started":1}
I20260812 06:59:22.526122 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.516853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.0.0.1:40636)
0812 06:59:22.517420 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:22.517439 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:22.517574 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:22.518818 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:22.518822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:22.518846 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:22.519203 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.519218 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.521351 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.521357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.522970 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.522983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.525012 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.525019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.525218 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.525274 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 06:59:22.525421 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 06:59:22.525631 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":1,"thread_start_us":272,"threads_started":1}
I20260812 06:59:22.526551 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.514525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:43509 (local address 127.0.0.1:43620)
0812 06:59:22.515352 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:22.515396 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:22.515557 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:22.517696 (+ 2139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:22.517704 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:22.517735 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:22.518175 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.518190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.520690 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.520698 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.523665 (+ 2967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.523680 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.525604 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.525611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.525764 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.525810 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 06:59:22.525879 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 06:59:22.526009 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":109,"thread_start_us":411,"threads_started":1}
I20260812 06:59:22.527386 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.514801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:40395 (local address 127.0.0.1:52224)
0812 06:59:22.516178 (+ 1377us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:22.516198 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:22.516312 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:22.518219 (+ 1907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:22.518224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:22.518254 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:22.518659 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.518673 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.520730 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.520736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.524272 (+ 3536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.524285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.526401 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.526411 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.526721 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.526766 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 06:59:22.526882 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 06:59:22.527009 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1216,"mutex_wait_us":191,"thread_start_us":1063,"threads_started":1}
I20260812 06:59:22.527576 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.514746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.26.226.3:43509)
0812 06:59:22.516978 (+ 2232us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.516990 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.517030 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:59:22.517148 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.517156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.517425 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.518406 (+ 981us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:22.518774 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.520544 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.523859 (+ 3315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.525331 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.526782 (+ 1451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.526988 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.527057 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.527204 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2029,"thread_start_us":2050,"threads_started":1}
I20260812 06:59:22.527961 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.515341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.26.226.2:40395)
0812 06:59:22.517528 (+ 2187us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.517536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.517563 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:59:22.517664 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.517673 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.517922 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.518200 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:22.518845 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.520567 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.524428 (+ 3861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.525728 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.525868 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.526060 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.526928 (+ 868us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.527090 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2013,"thread_start_us":1222,"threads_started":1}
I20260812 06:59:22.530716 18978 tablet_service.cc:1467] Processing CreateTablet for tablet 908814ae9d664ae89b4d534c2aa332c7 (DEFAULT_TABLE table=test-table [id=3f3562bdf4cb4780896cd3a9fcd4febd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:22.530969 19228 tablet_service.cc:1467] Processing CreateTablet for tablet 908814ae9d664ae89b4d534c2aa332c7 (DEFAULT_TABLE table=test-table [id=3f3562bdf4cb4780896cd3a9fcd4febd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:22.532570 18978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 908814ae9d664ae89b4d534c2aa332c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:22.532217 19465 tablet_service.cc:1467] Processing CreateTablet for tablet 908814ae9d664ae89b4d534c2aa332c7 (DEFAULT_TABLE table=test-table [id=3f3562bdf4cb4780896cd3a9fcd4febd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:22.533293 19228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 908814ae9d664ae89b4d534c2aa332c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:22.533666 19465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 908814ae9d664ae89b4d534c2aa332c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:22.544749 19597 tablet_bootstrap.cc:492] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: Bootstrap starting.
I20260812 06:59:22.547926 19601 tablet_bootstrap.cc:492] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: Bootstrap starting.
I20260812 06:59:22.549588 19597 tablet_bootstrap.cc:654] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:22.550197 19600 tablet_bootstrap.cc:492] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Bootstrap starting.
I20260812 06:59:22.551011 19597 log.cc:824] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:22.552687 19601 tablet_bootstrap.cc:654] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:22.553623 19600 tablet_bootstrap.cc:654] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:22.553808 19601 log.cc:824] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:22.554557 19597 tablet_bootstrap.cc:492] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: No bootstrap required, opened a new log
I20260812 06:59:22.554862 19597 ts_tablet_manager.cc:1397] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260812 06:59:22.555202 19600 log.cc:824] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:22.557615 19601 tablet_bootstrap.cc:492] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: No bootstrap required, opened a new log
I20260812 06:59:22.557920 19601 ts_tablet_manager.cc:1397] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260812 06:59:22.562953 19600 tablet_bootstrap.cc:492] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: No bootstrap required, opened a new log
I20260812 06:59:22.563261 19600 ts_tablet_manager.cc:1397] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260812 06:59:22.565905 19601 raft_consensus.cc:348] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } } peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } }
I20260812 06:59:22.566443 19601 raft_consensus.cc:374] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:22.566552 19601 raft_consensus.cc:729] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89275d8057949eb9c552e054bd53c79, State: Initialized, Role: FOLLOWER
I20260812 06:59:22.567134 19597 raft_consensus.cc:348] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } } peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } }
I20260812 06:59:22.567593 19597 raft_consensus.cc:374] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:22.567399 19601 consensus_queue.cc:260] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } } peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } }
I20260812 06:59:22.567706 19597 raft_consensus.cc:729] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5983c4830d8e4b91af2dde459d243f21, State: Initialized, Role: FOLLOWER
I20260812 06:59:22.568477 19597 consensus_queue.cc:260] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } } peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } }
I20260812 06:59:22.569643 19355 heartbeater.cc:503] Master 127.26.226.62:34519 was elected leader, sending a full tablet report...
I20260812 06:59:22.570895 19601 ts_tablet_manager.cc:1428] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260812 06:59:22.573164 19597 ts_tablet_manager.cc:1428] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
I20260812 06:59:22.574239 19600 raft_consensus.cc:348] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } } peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } }
I20260812 06:59:22.574681 19600 raft_consensus.cc:374] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:22.574779 19600 raft_consensus.cc:729] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d89bbfec43e4f30bd70dd2d6a6c78c7, State: Initialized, Role: FOLLOWER
I20260812 06:59:22.575384 19600 consensus_queue.cc:260] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } } peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } }
I20260812 06:59:22.576828 19568 heartbeater.cc:503] Master 127.26.226.62:34519 was elected leader, sending a full tablet report...
I20260812 06:59:22.577359 19600 ts_tablet_manager.cc:1428] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260812 06:59:22.589730 19606 raft_consensus.cc:482] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:22.590080 19606 raft_consensus.cc:504] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } } peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } }
I20260812 06:59:22.591943 19606 leader_election.cc:288] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d89bbfec43e4f30bd70dd2d6a6c78c7 (127.26.226.3:43509), 5983c4830d8e4b91af2dde459d243f21 (127.26.226.2:40395)
I20260812 06:59:22.599809 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.592432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:39157 (local address 127.26.226.3:43509)
0812 06:59:22.592876 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.592886 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.593584 (+ 698us) server_negotiation.cc:407] Connection header received
0812 06:59:22.593720 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.593726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.593801 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.593900 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:22.594614 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.595708 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.598035 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.598821 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.599019 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.599103 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.599407 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.599549 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253}
I20260812 06:59:22.599867 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.592586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:38139 (local address 127.26.226.2:40395)
0812 06:59:22.593127 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.593133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.593258 (+ 125us) server_negotiation.cc:407] Connection header received
0812 06:59:22.593500 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.593504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.593604 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.593730 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:22.594445 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.595554 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.598059 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.598874 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.599103 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.599181 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.599463 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.599598 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334}
I20260812 06:59:22.599987 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.592427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:43509 (local address 127.26.226.1:39157)
0812 06:59:22.593474 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:22.593492 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:22.593595 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:22.593938 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:22.593941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:22.593963 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:22.594446 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.594464 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.595880 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.595885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.597863 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.597878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.599062 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.599069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.599239 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.599264 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 06:59:22.599378 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 06:59:22.599520 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":44,"thread_start_us":655,"threads_started":1}
I20260812 06:59:22.600236 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.592271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:40395 (local address 127.26.226.1:38139)
0812 06:59:22.593109 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:22.593142 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:22.593344 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:22.593752 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:22.593755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:22.593777 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:22.594275 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.594292 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.595703 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.595711 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.597866 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:22.597882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.599118 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.599125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:22.599303 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.599327 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:59:22.599434 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 06:59:22.599579 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":28,"thread_start_us":444,"threads_started":1}
W20260812 06:59:22.600507 19358 tablet.cc:2369] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:22.601289 19492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908814ae9d664ae89b4d534c2aa332c7" candidate_uuid: "a89275d8057949eb9c552e054bd53c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" is_pre_election: true
I20260812 06:59:22.601468 19270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908814ae9d664ae89b4d534c2aa332c7" candidate_uuid: "a89275d8057949eb9c552e054bd53c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5983c4830d8e4b91af2dde459d243f21" is_pre_election: true
I20260812 06:59:22.601972 19270 raft_consensus.cc:2454] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a89275d8057949eb9c552e054bd53c79 in term 0.
I20260812 06:59:22.601969 19492 raft_consensus.cc:2454] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a89275d8057949eb9c552e054bd53c79 in term 0.
I20260812 06:59:22.602761 18903 leader_election.cc:302] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d89bbfec43e4f30bd70dd2d6a6c78c7, a89275d8057949eb9c552e054bd53c79; no voters:
I20260812 06:59:22.603233 19606 raft_consensus.cc:2781] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:22.603407 19606 raft_consensus.cc:482] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:22.603533 19606 raft_consensus.cc:3037] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:22.605979 19606 raft_consensus.cc:504] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } } peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } }
I20260812 06:59:22.606606 19606 leader_election.cc:288] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [CANDIDATE]: Term 1 election: Requested vote from peers 2d89bbfec43e4f30bd70dd2d6a6c78c7 (127.26.226.3:43509), 5983c4830d8e4b91af2dde459d243f21 (127.26.226.2:40395)
I20260812 06:59:22.607136 19492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908814ae9d664ae89b4d534c2aa332c7" candidate_uuid: "a89275d8057949eb9c552e054bd53c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7"
I20260812 06:59:22.607157 19270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908814ae9d664ae89b4d534c2aa332c7" candidate_uuid: "a89275d8057949eb9c552e054bd53c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5983c4830d8e4b91af2dde459d243f21"
I20260812 06:59:22.607353 19492 raft_consensus.cc:3037] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:22.607379 19270 raft_consensus.cc:3037] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:22.609719 19270 raft_consensus.cc:2454] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a89275d8057949eb9c552e054bd53c79 in term 1.
I20260812 06:59:22.609727 19492 raft_consensus.cc:2454] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a89275d8057949eb9c552e054bd53c79 in term 1.
I20260812 06:59:22.610200 18902 leader_election.cc:302] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5983c4830d8e4b91af2dde459d243f21, a89275d8057949eb9c552e054bd53c79; no voters:
I20260812 06:59:22.610584 19606 raft_consensus.cc:2781] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:22.611074 19606 raft_consensus.cc:686] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 1 LEADER]: Becoming Leader. State: Replica: a89275d8057949eb9c552e054bd53c79, State: Running, Role: LEADER
I20260812 06:59:22.611569 19606 consensus_queue.cc:237] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } } peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } }
I20260812 06:59:22.617053 18753 catalog_manager.cc:5495] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 reported cstate change: term changed from 0 to 1, leader changed from <none> to a89275d8057949eb9c552e054bd53c79 (127.26.226.1). New cstate: current_term: 1 leader_uuid: "a89275d8057949eb9c552e054bd53c79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } health_report { overall_health: HEALTHY } } }
I20260812 06:59:22.654495 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.648654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.26.226.1:43477)
0812 06:59:22.648878 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.648887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.648912 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:59:22.649207 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.649212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.649296 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.649462 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:22.649870 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.650604 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.652152 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.652574 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.652717 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.653295 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:22.653830 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:22.654024 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.654131 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.654220 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:59:22.663247 19270 raft_consensus.cc:1261] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21 [term 1 FOLLOWER]: Refusing update from remote peer a89275d8057949eb9c552e054bd53c79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:22.663254 19492 raft_consensus.cc:1261] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 1 FOLLOWER]: Refusing update from remote peer a89275d8057949eb9c552e054bd53c79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:22.664356 19611 consensus_queue.cc:1035] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:22.664726 19606 consensus_queue.cc:1035] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:22.677053 19622 mvcc.cc:205] Tried to move back new op lower bound from 7317577575058345984 to 7317577574861123584. Current Snapshot: MvccSnapshot[applied={T|T < 7317577575058345984}]
I20260812 06:59:22.687005 19625 mvcc.cc:205] Tried to move back new op lower bound from 7317577575058345984 to 7317577574861123584. Current Snapshot: MvccSnapshot[applied={T|T < 7317577575058345984}]
I20260812 06:59:22.695852 18752 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:22.696202 18752 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:22.700196 18752 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Sending DeleteTablet for 3 replicas of tablet 908814ae9d664ae89b4d534c2aa332c7
I20260812 06:59:22.701423 19465 tablet_service.cc:1514] Processing DeleteTablet for tablet 908814ae9d664ae89b4d534c2aa332c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:22 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:43620
I20260812 06:59:22.701453 18978 tablet_service.cc:1514] Processing DeleteTablet for tablet 908814ae9d664ae89b4d534c2aa332c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:22 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:40636
I20260812 06:59:22.701453 19228 tablet_service.cc:1514] Processing DeleteTablet for tablet 908814ae9d664ae89b4d534c2aa332c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:22 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:52224
I20260812 06:59:22.702798 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 18870
I20260812 06:59:22.702867 19640 tablet_replica.cc:330] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: stopping tablet replica
I20260812 06:59:22.703430 19640 raft_consensus.cc:2229] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:22.704046 19640 raft_consensus.cc:2258] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:22.707310 19640 ts_tablet_manager.cc:1905] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:22.717226 19640 ts_tablet_manager.cc:1918] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:22.717533 19640 log.cc:1196] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/908814ae9d664ae89b4d534c2aa332c7
I20260812 06:59:22.718251 19640 ts_tablet_manager.cc:1939] T 908814ae9d664ae89b4d534c2aa332c7 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Deleting consensus metadata
I20260812 06:59:22.720187 19644 tablet_replica.cc:330] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: stopping tablet replica
I20260812 06:59:22.720834 19644 raft_consensus.cc:2229] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:22.721048 18736 catalog_manager.cc:4841] TS 2d89bbfec43e4f30bd70dd2d6a6c78c7 (127.26.226.3:43509): tablet 908814ae9d664ae89b4d534c2aa332c7 (table test-table [id=3f3562bdf4cb4780896cd3a9fcd4febd]) successfully deleted
I20260812 06:59:22.721383 19644 raft_consensus.cc:2258] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:22.723839 19644 ts_tablet_manager.cc:1905] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:22.724507 19643 tablet_replica.cc:330] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: stopping tablet replica
I20260812 06:59:22.724972 19643 raft_consensus.cc:2229] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:59:22.725625 19643 raft_consensus.cc:2258] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:22.727816 19643 ts_tablet_manager.cc:1905] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 06:59:22 dist-test-slave-t3q3 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:59:22.736620 19643 ts_tablet_manager.cc:1918] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:22.736855 19643 log.cc:1196] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/908814ae9d664ae89b4d534c2aa332c7
I20260812 06:59:22.737452 19643 ts_tablet_manager.cc:1939] T 908814ae9d664ae89b4d534c2aa332c7 P a89275d8057949eb9c552e054bd53c79: Deleting consensus metadata
I20260812 06:59:22.739509 18737 catalog_manager.cc:4841] TS a89275d8057949eb9c552e054bd53c79 (127.26.226.1:43477): tablet 908814ae9d664ae89b4d534c2aa332c7 (table test-table [id=3f3562bdf4cb4780896cd3a9fcd4febd]) successfully deleted
I20260812 06:59:22.740430 19644 ts_tablet_manager.cc:1918] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:22.740658 19644 log.cc:1196] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/908814ae9d664ae89b4d534c2aa332c7
I20260812 06:59:22.741250 19644 ts_tablet_manager.cc:1939] T 908814ae9d664ae89b4d534c2aa332c7 P 5983c4830d8e4b91af2dde459d243f21: Deleting consensus metadata
I20260812 06:59:22.743258 18735 catalog_manager.cc:4841] TS 5983c4830d8e4b91af2dde459d243f21 (127.26.226.2:40395): tablet 908814ae9d664ae89b4d534c2aa332c7 (table test-table [id=3f3562bdf4cb4780896cd3a9fcd4febd]) successfully deleted
I20260812 06:59:22.743777 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.704101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40648 (local address 127.26.226.1:43477)
0812 06:59:22.704298 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.704305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.717394 (+ 13089us) server_negotiation.cc:407] Connection header received
0812 06:59:22.718775 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.718784 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.718910 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.719024 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:22.721969 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.723719 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.725899 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.726908 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.727036 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.734995 (+ 7959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:22.735012 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:22.735035 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:22.735103 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:22.739923 (+ 4820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:22.741062 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:22.741075 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:22.741082 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:22.741316 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:22.741894 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:22.741900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:22.741903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:22.742394 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:22.742741 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.743089 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.743379 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:59:22.744594 19094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:22.844651 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 18870
I20260812 06:59:22.880187 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 19115
Aug 12 06:59:22 dist-test-slave-t3q3 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:59:22.897445 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.881192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.26.226.2:40395)
0812 06:59:22.881405 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.881412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.881433 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:59:22.881508 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.881513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.881589 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.881705 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:22.883029 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.884323 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.885731 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:22.886403 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:22.886529 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:22.891062 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:22.891076 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:22.891092 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:22.891131 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:22.894299 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:22.895058 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:22.895066 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:22.895069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:22.895201 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:22.895622 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:22.895627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:22.895631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:22.896037 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:22.896539 (+ 502us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.896836 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.897019 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 06:59:22.898241 19346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:22.992620 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 19115
I20260812 06:59:23.023641 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 19363
Aug 12 06:59:23 dist-test-slave-t3q3 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:59:23.040576 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:23.024569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43634 (local address 127.26.226.3:43509)
0812 06:59:23.024776 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:23.024783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:23.024806 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:59:23.025018 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:23.025023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:23.025089 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:23.025211 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:23.026639 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:23.027604 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:23.029086 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:23.029519 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:23.029668 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:23.034781 (+ 5113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:23.034815 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:23.034832 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:23.034870 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:23.037804 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:23.038584 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:23.038592 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:23.038594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:23.038724 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:23.039202 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:23.039206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:23.039208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:23.039565 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:23.039803 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:23.040057 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 06:59:23.040315 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:59:23.041278 19562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:23.131706 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 19363
I20260812 06:59:23.155790 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 18707
I20260812 06:59:23.157437 18813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:23.270555 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 18707
I20260812 06:59:23.303198 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34519
--webserver_interface=127.26.226.62
--webserver_port=43207
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:34519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:23.656421 19650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:23.656790 19650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:23.656965 19650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:23.664269 19650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:23.664443 19650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:23.664561 19650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:23.664669 19650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:23.664763 19650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:23.674862 19650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:34519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=43207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:23.676308 19650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:23.677664 19650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:23.704926 19660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:23.704954 19661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:23.704926 19663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:23.706333 19650 server_base.cc:1034] running on GCE node
I20260812 06:59:23.707326 19650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:23.709070 19650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:23.710327 19650 hybrid_clock.cc:648] HybridClock initialized: now 1786517963710281 us; error 58 us; skew 500 ppm
I20260812 06:59:23.710753 19650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:23 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517963, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:23.714596 19650 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:59:23.716384 19650 webserver.cc:460] Webserver started at http://127.26.226.62:43207/ using document root <none> and password file <none>
I20260812 06:59:23.717175 19650 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:23.717331 19650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:23.723456 19650 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 06:59:23.726626 19674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:23.728052 19650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:59:23.728319 19650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "42d525d1eb154a16ae71ced739fea360"
format_stamp: "Formatted at 2026-08-12 06:59:20 on dist-test-slave-t3q3"
server_key: "8b7a27faa52ebc95fc205c5596058ac5"
server_key_iv: "b72264e01947e8c04a91847900f05367"
server_key_version: "encryptionkey@0"
I20260812 06:59:23.729423 19650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:23.747246 19650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:23.751849 19650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:23.752200 19650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:23.772430 19650 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:34519
I20260812 06:59:23.772454 19757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:34519 every 8 connection(s)
I20260812 06:59:23.774070 19650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:23.781559 19758 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:59:23.783622 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 19650
I20260812 06:59:23.784559 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:43477
--local_ip_for_outbound_sockets=127.26.226.1
--tserver_master_addrs=127.26.226.62:34519
--webserver_port=38529
--webserver_interface=127.26.226.1
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:23.785121 19758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Bootstrap starting.
I20260812 06:59:23.805153 19758 log.cc:824] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:23.817632 19758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:59:23.818240 19758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Bootstrap complete.
I20260812 06:59:23.828011 19758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:23.829030 19758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42d525d1eb154a16ae71ced739fea360, State: Initialized, Role: FOLLOWER
I20260812 06:59:23.829754 19758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:23.830013 19758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:23.830147 19758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:23.830344 19758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:59:23.834751 19758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:23.835345 19758 leader_election.cc:302] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42d525d1eb154a16ae71ced739fea360; no voters:
I20260812 06:59:23.836048 19758 leader_election.cc:288] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:59:23.836223 19763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:59:23.837798 19763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 2 LEADER]: Becoming Leader. State: Replica: 42d525d1eb154a16ae71ced739fea360, State: Running, Role: LEADER
I20260812 06:59:23.838438 19763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:23.838752 19758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:23.841144 19772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42d525d1eb154a16ae71ced739fea360. Latest consensus state: current_term: 2 leader_uuid: "42d525d1eb154a16ae71ced739fea360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } } }
I20260812 06:59:23.841231 19766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "42d525d1eb154a16ae71ced739fea360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } } }
I20260812 06:59:23.841655 19766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:23.841542 19772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:23.855103 19780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:23.863966 19780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:23.865492 19780 catalog_manager.cc:1259] Loaded cluster ID: 7c1a1b584a6447ca933aaab8c40e3ea3
I20260812 06:59:23.865667 19780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:23.869123 19780 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 06:59:23.870712 19797 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 06:59:23.886785 19780 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 06:59:23.896759 19780 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:23.898581 19780 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Loaded TSK: 0
I20260812 06:59:23.900242 19780 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:59:24.168601 19760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:24.168905 19760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:24.169057 19760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:24.176926 19760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:24.177055 19760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:24.177255 19760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:59:24.187695 19760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:43477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=38529
--tserver_master_addrs=127.26.226.62:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:24.189123 19760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:24.190546 19760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:24.205652 19806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:24.205633 19807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:24.207049 19760 server_base.cc:1034] running on GCE node
W20260812 06:59:24.207123 19809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:24.207836 19760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:24.208904 19760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:24.210176 19760 hybrid_clock.cc:648] HybridClock initialized: now 1786517964210114 us; error 49 us; skew 500 ppm
I20260812 06:59:24.210548 19760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:24 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517964, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:24.214118 19760 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:59:24.215963 19760 webserver.cc:460] Webserver started at http://127.26.226.1:38529/ using document root <none> and password file <none>
I20260812 06:59:24.216713 19760 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:24.216882 19760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:24.222443 19760 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:24.225380 19817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:24.226872 19760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:59:24.227061 19760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "a89275d8057949eb9c552e054bd53c79"
format_stamp: "Formatted at 2026-08-12 06:59:21 on dist-test-slave-t3q3"
server_key: "44e3ea896e40631072f3ee570f6a77b7"
server_key_iv: "c23ce8e53efc425f4f8928ac74c01921"
server_key_version: "encryptionkey@0"
I20260812 06:59:24.228113 19760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:24.247313 19760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:24.251959 19760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:24.252307 19760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:24.253412 19760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:24.254808 19760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:24.254945 19760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:24.255079 19760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:24.255177 19760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:24.290951 19760 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:43477
I20260812 06:59:24.291014 20002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:43477 every 8 connection(s)
I20260812 06:59:24.292498 19760 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:24.297003 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 19760
I20260812 06:59:24.297662 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:40395
--local_ip_for_outbound_sockets=127.26.226.2
--tserver_master_addrs=127.26.226.62:34519
--webserver_port=39435
--webserver_interface=127.26.226.2
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:59:24 dist-test-slave-t3q3 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517964, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:24.318861 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:24.295536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:33799 (local address 127.26.226.62:34519)
0812 06:59:24.296186 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:24.296198 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:24.297971 (+ 1773us) server_negotiation.cc:407] Connection header received
0812 06:59:24.299343 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:24.299364 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:24.299826 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:24.300236 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:24.301674 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.303682 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.305392 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.306504 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.306732 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:24.310778 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:24.310787 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:24.310801 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:24.310848 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:24.313983 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:24.314997 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:24.315007 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:24.315014 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:24.315180 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:24.315734 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:24.315738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:24.315740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:24.316325 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:24.316480 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:24.317514 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0812 06:59:24.317863 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":231,"threads_started":1}
I20260812 06:59:24.319321 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:24.295839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:33799)
0812 06:59:24.296792 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:24.296839 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:24.298694 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:24.300349 (+ 1655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:24.300356 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:24.300874 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:24.301410 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:24.301428 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.303895 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.303900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:24.305228 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:24.305243 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.306749 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.306754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:24.307080 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:24.307827 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:24.307855 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:24.310471 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:24.314159 (+ 3688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:24.314171 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:24.314190 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:24.314841 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:24.315308 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:24.315316 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:24.315323 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:24.315575 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:24.316473 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:24.316485 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:24.316926 (+ 441us) client_negotiation.cc:769] Sending connection context
0812 06:59:24.317335 (+ 409us) client_negotiation.cc:240] Negotiation successful
0812 06:59:24.317945 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":441,"threads_started":1}
I20260812 06:59:24.322369 20007 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34519
I20260812 06:59:24.322723 20007 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:24.323767 20007 heartbeater.cc:511] Master 127.26.226.62:34519 requested a full tablet report, sending...
I20260812 06:59:24.326435 19703 ts_manager.cc:194] Registered new tserver with Master: a89275d8057949eb9c552e054bd53c79 (127.26.226.1:43477)
I20260812 06:59:24.327869 19703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:33799
W20260812 06:59:24.662024 20014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:24.662335 20014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:24.662500 20014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:24.670013 20014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:24.670149 20014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:24.670326 20014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:59:24.681164 20014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:40395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=39435
--tserver_master_addrs=127.26.226.62:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:24.682634 20014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:24.684202 20014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:24.698602 20026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:24.698654 20024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:24.698654 20022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:24.700162 20014 server_base.cc:1034] running on GCE node
I20260812 06:59:24.700940 20014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:24.702144 20014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:24.703408 20014 hybrid_clock.cc:648] HybridClock initialized: now 1786517964703353 us; error 51 us; skew 500 ppm
I20260812 06:59:24.703742 20014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:24 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517964, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:24.707513 20014 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:59:24.709275 20014 webserver.cc:460] Webserver started at http://127.26.226.2:39435/ using document root <none> and password file <none>
I20260812 06:59:24.710083 20014 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:24.710264 20014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:24.716944 20014 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:59:24.719919 20034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:24.721431 20014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:59:24.721632 20014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "5983c4830d8e4b91af2dde459d243f21"
format_stamp: "Formatted at 2026-08-12 06:59:21 on dist-test-slave-t3q3"
server_key: "bc2bc8287b54a2917ae279d4e5623b4e"
server_key_iv: "a30af246a7d1df448779c6129f9daa04"
server_key_version: "encryptionkey@0"
I20260812 06:59:24.722651 20014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:24.751582 20014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:24.756479 20014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:24.756845 20014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:24.757937 20014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:24.759342 20014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:24.759444 20014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:24.759522 20014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:24.759624 20014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:24.800354 20014 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:40395
I20260812 06:59:24.800449 20217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:40395 every 8 connection(s)
I20260812 06:59:24.801936 20014 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:24.804112 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 20014
I20260812 06:59:24.804770 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:43509
--local_ip_for_outbound_sockets=127.26.226.3
--tserver_master_addrs=127.26.226.62:34519
--webserver_port=45585
--webserver_interface=127.26.226.3
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:59:24 dist-test-slave-t3q3 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517964, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:24.823897 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:24.805214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:53571 (local address 127.26.226.62:34519)
0812 06:59:24.805446 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:24.805454 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:24.806751 (+ 1297us) server_negotiation.cc:407] Connection header received
0812 06:59:24.808827 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:24.808836 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:24.808949 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:24.809075 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:24.810661 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.811368 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.813163 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.813627 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.813775 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:24.817616 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:24.817623 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:24.817626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:24.817665 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:24.820484 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:24.821340 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:24.821345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:24.821347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:24.821490 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:24.821982 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:24.821986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:24.821988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:24.822283 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:24.822469 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:24.823258 (+ 789us) server_negotiation.cc:299] Negotiation successful
0812 06:59:24.823526 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 06:59:24.825053 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:24.805649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:53571)
0812 06:59:24.806565 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:24.806622 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:24.808536 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:24.809331 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:24.809339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:24.809885 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:24.810475 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:24.810492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.811551 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.811561 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:24.812988 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:24.813002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.813873 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.813881 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:24.814131 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:24.814894 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:24.814919 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:24.817365 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:24.820658 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:24.820668 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:24.820687 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:24.821187 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:24.821624 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:24.821629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:24.821633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:24.821842 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:24.822428 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:24.822439 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:24.822833 (+ 394us) client_negotiation.cc:769] Sending connection context
0812 06:59:24.823128 (+ 295us) client_negotiation.cc:240] Negotiation successful
0812 06:59:24.823749 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":424,"threads_started":1}
I20260812 06:59:24.826984 20218 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34519
I20260812 06:59:24.827404 20218 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:24.828370 20218 heartbeater.cc:511] Master 127.26.226.62:34519 requested a full tablet report, sending...
I20260812 06:59:24.830103 19703 ts_manager.cc:194] Registered new tserver with Master: 5983c4830d8e4b91af2dde459d243f21 (127.26.226.2:40395)
I20260812 06:59:24.831064 19703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:53571
W20260812 06:59:25.165756 20226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:25.166090 20226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:25.166271 20226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:25.174199 20226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:25.174326 20226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:25.174525 20226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:59:25.185182 20226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:43509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=45585
--tserver_master_addrs=127.26.226.62:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:25.186620 20226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:25.188112 20226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:25.202880 20234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:25.202924 20231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:25.202880 20237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:25.208799 20226 server_base.cc:1034] running on GCE node
I20260812 06:59:25.209630 20226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:25.210716 20226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:25.211969 20226 hybrid_clock.cc:648] HybridClock initialized: now 1786517965211914 us; error 52 us; skew 500 ppm
I20260812 06:59:25.212338 20226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:25 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517965, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:25.215817 20226 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:59:25.217621 20226 webserver.cc:460] Webserver started at http://127.26.226.3:45585/ using document root <none> and password file <none>
I20260812 06:59:25.218397 20226 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:25.218572 20226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:25.224624 20226 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:59:25.227908 20248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:25.229475 20226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:25.229704 20226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7"
format_stamp: "Formatted at 2026-08-12 06:59:22 on dist-test-slave-t3q3"
server_key: "31f796a6f63903e76dae46e51df1c5be"
server_key_iv: "f0beb68f6450b4a08d16cf8b9b2604cf"
server_key_version: "encryptionkey@0"
I20260812 06:59:25.230829 20226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:25.250131 20226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:25.255162 20226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:25.255437 20226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:25.256661 20226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:25.258330 20226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:25.258441 20226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:25.258616 20226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:25.258715 20226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:25.295825 20226 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:43509
I20260812 06:59:25.295892 20416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:43509 every 8 connection(s)
I20260812 06:59:25.297389 20226 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:25.299603 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 20226
Aug 12 06:59:25 dist-test-slave-t3q3 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517965, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:25.318080 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.300554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.26.226.62:34519)
0812 06:59:25.301074 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.301082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.301677 (+ 595us) server_negotiation.cc:407] Connection header received
0812 06:59:25.301988 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.301993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.302077 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.302207 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:25.304476 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.305447 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.307400 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.308171 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.308303 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.311528 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:25.311535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:25.311540 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:25.311582 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:25.314873 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:25.315600 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:25.315608 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:25.315614 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:25.315763 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:25.316226 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:25.316232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:25.316235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:25.316634 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:25.316927 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.317304 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.317604 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":389,"thread_start_us":248,"threads_started":1}
I20260812 06:59:25.322122 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.300397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:52279 (local address 127.26.226.62:34519)
0812 06:59:25.300635 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.300643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.302240 (+ 1597us) server_negotiation.cc:407] Connection header received
0812 06:59:25.304984 (+ 2744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.304989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.305080 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.305192 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:25.307203 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.308443 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.310926 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.311409 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.311544 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.315209 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:25.315214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:25.315217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:25.315247 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:25.318633 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:25.319509 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:25.319513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:25.319515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:25.319699 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:25.320265 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:25.320270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:25.320273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:25.320611 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:25.320768 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.321646 (+ 878us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.321832 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":83}
I20260812 06:59:25.323087 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.301077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:52279)
0812 06:59:25.302073 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.302117 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:25.304723 (+ 2606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:25.305411 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:25.305420 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:25.306204 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:25.306994 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.307021 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.308613 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.308620 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.310770 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.310789 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.311647 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.311652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.311864 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.312536 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:25.312560 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:25.314994 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:25.318780 (+ 3786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:25.318790 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:25.318807 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:25.319389 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:25.319824 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:25.319829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:25.319832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:25.320146 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:25.320735 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:25.320741 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:25.321177 (+ 436us) client_negotiation.cc:769] Sending connection context
0812 06:59:25.321501 (+ 324us) client_negotiation.cc:240] Negotiation successful
0812 06:59:25.321915 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":450,"threads_started":1}
I20260812 06:59:25.324751 20417 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34519
I20260812 06:59:25.325139 20417 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:25.326058 20417 heartbeater.cc:511] Master 127.26.226.62:34519 requested a full tablet report, sending...
I20260812 06:59:25.327942 19703 ts_manager.cc:194] Registered new tserver with Master: 2d89bbfec43e4f30bd70dd2d6a6c78c7 (127.26.226.3:43509)
I20260812 06:59:25.328910 19703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:52279
I20260812 06:59:25.331984 20007 heartbeater.cc:503] Master 127.26.226.62:34519 was elected leader, sending a full tablet report...
I20260812 06:59:25.334844 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:25.342691 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.335911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40420 (local address 127.26.226.62:34519)
0812 06:59:25.336250 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.336260 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.336617 (+ 357us) server_negotiation.cc:407] Connection header received
0812 06:59:25.336879 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.336884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.337029 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.337195 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:25.337665 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.338451 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.340291 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.340800 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.340944 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.341521 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:25.342064 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:25.342233 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.342324 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.342412 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260812 06:59:25.343847 19701 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:59:25.351610 19701 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:59:25.387591 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.365936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.26.226.1:43477)
0812 06:59:25.378039 (+ 12103us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.378047 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.378694 (+ 647us) server_negotiation.cc:407] Connection header received
0812 06:59:25.379010 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.379021 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.379277 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.379453 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:25.380481 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.381998 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.384742 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.386304 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.386460 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.386679 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.387025 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.387220 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11908,"thread_start_us":9985,"threads_started":1}
I20260812 06:59:25.387630 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.365110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.0.0.1:40662)
0812 06:59:25.378577 (+ 13467us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.378597 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:25.378815 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:25.379777 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:25.379781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:25.379806 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:25.380292 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.380302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.382191 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.382197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.384543 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.384560 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.386535 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.386541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.386769 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.386801 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:59:25.386899 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 06:59:25.387189 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13327,"mutex_wait_us":125,"thread_start_us":13159,"threads_started":1}
I20260812 06:59:25.388592 20433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.364815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.26.226.2:40395)
0812 06:59:25.365623 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.365633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.377453 (+ 11820us) server_negotiation.cc:407] Connection header received
0812 06:59:25.377958 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.377967 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.378287 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.378477 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:25.380023 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.381487 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.384789 (+ 3302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.385873 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.386022 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.386254 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.387992 (+ 1738us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.388215 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":1572,"threads_started":1}
I20260812 06:59:25.388562 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.364601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:40395 (local address 127.0.0.1:52242)
0812 06:59:25.377276 (+ 12675us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.377324 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:25.377646 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:25.378866 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:25.378875 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:25.378905 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:25.379777 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.379789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.381666 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.381673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.384548 (+ 2875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.384560 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.386403 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.386412 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.386636 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.386740 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 06:59:25.386908 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 06:59:25.388146 (+ 1238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12376,"mutex_wait_us":1017,"thread_start_us":15385,"threads_started":1}
I20260812 06:59:25.389420 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.366039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43636 (local address 127.26.226.3:43509)
0812 06:59:25.376950 (+ 10911us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.376964 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.380048 (+ 3084us) server_negotiation.cc:407] Connection header received
0812 06:59:25.380599 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.380608 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.380832 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.381047 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:25.381712 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.383363 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.386006 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.387495 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.387630 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.387870 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.388440 (+ 570us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.388620 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10681,"thread_start_us":10285,"threads_started":1}
I20260812 06:59:25.390089 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.365270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:43509 (local address 127.0.0.1:43636)
0812 06:59:25.379888 (+ 14618us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.379904 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:25.380313 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:25.381063 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:25.381068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:25.381095 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:25.381463 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.381473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.383512 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.383518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.385792 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.385810 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.387951 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.387959 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.388216 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.388265 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 06:59:25.388365 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 06:59:25.389699 (+ 1334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14474,"mutex_wait_us":43,"thread_start_us":11716,"threads_started":1}
I20260812 06:59:25.393262 19900 tablet_service.cc:1467] Processing CreateTablet for tablet 01c467373a404494be1d19fb8a08b505 (DEFAULT_TABLE table=test-table [id=ab8192528650450097473206dfc93e21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:25.393575 20323 tablet_service.cc:1467] Processing CreateTablet for tablet 01c467373a404494be1d19fb8a08b505 (DEFAULT_TABLE table=test-table [id=ab8192528650450097473206dfc93e21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:25.394625 19900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01c467373a404494be1d19fb8a08b505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:25.394783 20323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01c467373a404494be1d19fb8a08b505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:25.399024 20112 tablet_service.cc:1467] Processing CreateTablet for tablet 01c467373a404494be1d19fb8a08b505 (DEFAULT_TABLE table=test-table [id=ab8192528650450097473206dfc93e21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:25.401448 20112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01c467373a404494be1d19fb8a08b505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:25.408417 20440 tablet_bootstrap.cc:492] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Bootstrap starting.
I20260812 06:59:25.411288 20442 tablet_bootstrap.cc:492] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Bootstrap starting.
I20260812 06:59:25.413036 20440 tablet_bootstrap.cc:654] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:25.414372 20440 log.cc:824] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:25.414925 20442 tablet_bootstrap.cc:654] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:25.416290 20442 log.cc:824] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:25.418099 20440 tablet_bootstrap.cc:492] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: No bootstrap required, opened a new log
I20260812 06:59:25.418397 20440 ts_tablet_manager.cc:1397] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 06:59:25.419749 20442 tablet_bootstrap.cc:492] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: No bootstrap required, opened a new log
I20260812 06:59:25.420011 20442 ts_tablet_manager.cc:1397] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260812 06:59:25.424470 20445 tablet_bootstrap.cc:492] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: Bootstrap starting.
I20260812 06:59:25.428463 20445 tablet_bootstrap.cc:654] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:25.429571 20445 log.cc:824] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:25.430790 20440 raft_consensus.cc:348] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } } peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } }
I20260812 06:59:25.431295 20440 raft_consensus.cc:374] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:25.431413 20440 raft_consensus.cc:729] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89275d8057949eb9c552e054bd53c79, State: Initialized, Role: FOLLOWER
I20260812 06:59:25.431125 20442 raft_consensus.cc:348] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } } peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } }
I20260812 06:59:25.431564 20442 raft_consensus.cc:374] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:25.431674 20442 raft_consensus.cc:729] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d89bbfec43e4f30bd70dd2d6a6c78c7, State: Initialized, Role: FOLLOWER
I20260812 06:59:25.432294 20442 consensus_queue.cc:260] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } } peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } }
I20260812 06:59:25.432296 20440 consensus_queue.cc:260] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } } peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } }
I20260812 06:59:25.433028 20445 tablet_bootstrap.cc:492] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: No bootstrap required, opened a new log
I20260812 06:59:25.433565 20445 ts_tablet_manager.cc:1397] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 06:59:25.434369 20442 ts_tablet_manager.cc:1428] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260812 06:59:25.434676 20440 ts_tablet_manager.cc:1428] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260812 06:59:25.435348 20417 heartbeater.cc:503] Master 127.26.226.62:34519 was elected leader, sending a full tablet report...
I20260812 06:59:25.444162 20445 raft_consensus.cc:348] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } } peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } }
I20260812 06:59:25.444653 20445 raft_consensus.cc:374] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:25.444774 20445 raft_consensus.cc:729] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5983c4830d8e4b91af2dde459d243f21, State: Initialized, Role: FOLLOWER
I20260812 06:59:25.445506 20445 consensus_queue.cc:260] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } } peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } }
I20260812 06:59:25.446769 20218 heartbeater.cc:503] Master 127.26.226.62:34519 was elected leader, sending a full tablet report...
I20260812 06:59:25.447530 20445 ts_tablet_manager.cc:1428] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: Time spent starting tablet: real 0.014s user 0.011s sys 0.000s
I20260812 06:59:25.513974 20452 raft_consensus.cc:482] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:25.514334 20452 raft_consensus.cc:504] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } } peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } }
I20260812 06:59:25.516108 20452 leader_election.cc:288] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89275d8057949eb9c552e054bd53c79 (127.26.226.1:43477), 2d89bbfec43e4f30bd70dd2d6a6c78c7 (127.26.226.3:43509)
I20260812 06:59:25.523540 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.516621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:41983 (local address 127.26.226.3:43509)
0812 06:59:25.516842 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.516865 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.517773 (+ 908us) server_negotiation.cc:407] Connection header received
0812 06:59:25.517991 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.517998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.518094 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.518200 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:25.518892 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.519736 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.521700 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.522514 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.522712 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.522788 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.523130 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.523266 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:59:25.523576 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.516606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:37727 (local address 127.26.226.1:43477)
0812 06:59:25.516834 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.516843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.518014 (+ 1171us) server_negotiation.cc:407] Connection header received
0812 06:59:25.518216 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.518221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.518298 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.518405 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:25.519084 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.519848 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.521746 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.522502 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.522691 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.522782 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.523179 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.523307 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:59:25.523743 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.516981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.26.226.2:37727)
0812 06:59:25.517887 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.517907 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:25.518054 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:25.518432 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:25.518435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:25.518459 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:25.518913 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.518927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.520009 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.520013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.521557 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.521571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.522750 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.522760 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.522938 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.523011 (+ 73us) client_negotiation.cc:769] Sending connection context
0812 06:59:25.523131 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 06:59:25.523266 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":47,"thread_start_us":518,"threads_started":1}
I20260812 06:59:25.523773 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.516619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:43509 (local address 127.26.226.2:41983)
0812 06:59:25.517614 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.517648 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:25.517813 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:25.518224 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:25.518229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:25.518254 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:25.518717 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.518727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.519880 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.519888 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.521521 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:25.521535 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.522775 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.522781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:25.522943 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.522986 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 06:59:25.523064 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 06:59:25.523230 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":31,"thread_start_us":507,"threads_started":1}
I20260812 06:59:25.525063 20344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01c467373a404494be1d19fb8a08b505" candidate_uuid: "5983c4830d8e4b91af2dde459d243f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" is_pre_election: true
I20260812 06:59:25.525045 19933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01c467373a404494be1d19fb8a08b505" candidate_uuid: "5983c4830d8e4b91af2dde459d243f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89275d8057949eb9c552e054bd53c79" is_pre_election: true
I20260812 06:59:25.525580 19933 raft_consensus.cc:2454] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5983c4830d8e4b91af2dde459d243f21 in term 0.
I20260812 06:59:25.525586 20344 raft_consensus.cc:2454] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5983c4830d8e4b91af2dde459d243f21 in term 0.
I20260812 06:59:25.526304 20039 leader_election.cc:302] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5983c4830d8e4b91af2dde459d243f21, a89275d8057949eb9c552e054bd53c79; no voters:
I20260812 06:59:25.526742 20452 raft_consensus.cc:2781] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:25.526894 20452 raft_consensus.cc:482] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:25.527001 20452 raft_consensus.cc:3037] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:25.529527 20452 raft_consensus.cc:504] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } } peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } }
I20260812 06:59:25.530221 20452 leader_election.cc:288] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [CANDIDATE]: Term 1 election: Requested vote from peers a89275d8057949eb9c552e054bd53c79 (127.26.226.1:43477), 2d89bbfec43e4f30bd70dd2d6a6c78c7 (127.26.226.3:43509)
I20260812 06:59:25.530692 19933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01c467373a404494be1d19fb8a08b505" candidate_uuid: "5983c4830d8e4b91af2dde459d243f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89275d8057949eb9c552e054bd53c79"
I20260812 06:59:25.530745 20344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01c467373a404494be1d19fb8a08b505" candidate_uuid: "5983c4830d8e4b91af2dde459d243f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7"
I20260812 06:59:25.530925 19933 raft_consensus.cc:3037] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:25.530957 20344 raft_consensus.cc:3037] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:25.533557 20344 raft_consensus.cc:2454] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5983c4830d8e4b91af2dde459d243f21 in term 1.
I20260812 06:59:25.533697 19933 raft_consensus.cc:2454] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5983c4830d8e4b91af2dde459d243f21 in term 1.
I20260812 06:59:25.534072 20037 leader_election.cc:302] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d89bbfec43e4f30bd70dd2d6a6c78c7, 5983c4830d8e4b91af2dde459d243f21; no voters:
I20260812 06:59:25.534427 20452 raft_consensus.cc:2781] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:25.534929 20452 raft_consensus.cc:686] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 1 LEADER]: Becoming Leader. State: Replica: 5983c4830d8e4b91af2dde459d243f21, State: Running, Role: LEADER
I20260812 06:59:25.535528 20452 consensus_queue.cc:237] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } } peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } }
I20260812 06:59:25.540442 19698 catalog_manager.cc:5495] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5983c4830d8e4b91af2dde459d243f21 (127.26.226.2). New cstate: current_term: 1 leader_uuid: "5983c4830d8e4b91af2dde459d243f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4830d8e4b91af2dde459d243f21" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 40395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 } health_report { overall_health: UNKNOWN } } }
W20260812 06:59:25.545149 20011 tablet.cc:2369] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:59:25.549105 20418 tablet.cc:2369] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:59:25.554097 20219 tablet.cc:2369] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:25.595180 20433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.589143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52252 (local address 127.26.226.2:40395)
0812 06:59:25.589395 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.589404 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.589426 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:59:25.589544 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.589548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.589645 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.589827 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:25.590254 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.590999 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.592529 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.592928 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.593092 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.593649 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:25.594448 (+ 799us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:25.594621 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.594775 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.594868 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 06:59:25.604140 19933 raft_consensus.cc:1261] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79 [term 1 FOLLOWER]: Refusing update from remote peer 5983c4830d8e4b91af2dde459d243f21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:25.604218 20344 raft_consensus.cc:1261] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 1 FOLLOWER]: Refusing update from remote peer 5983c4830d8e4b91af2dde459d243f21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:25.605334 20452 consensus_queue.cc:1035] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89275d8057949eb9c552e054bd53c79" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 43477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:25.605654 20456 consensus_queue.cc:1035] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:25.617352 20464 mvcc.cc:205] Tried to move back new op lower bound from 7317577587102408704 to 7317577586838142976. Current Snapshot: MvccSnapshot[applied={T|T < 7317577587102408704}]
I20260812 06:59:25.618992 20465 mvcc.cc:205] Tried to move back new op lower bound from 7317577587102408704 to 7317577586838142976. Current Snapshot: MvccSnapshot[applied={T|T < 7317577587102408704}]
I20260812 06:59:25.635457 19698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:25.635839 19698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:25.639993 19698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Sending DeleteTablet for 3 replicas of tablet 01c467373a404494be1d19fb8a08b505
I20260812 06:59:25.641394 20112 tablet_service.cc:1514] Processing DeleteTablet for tablet 01c467373a404494be1d19fb8a08b505 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:25 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:52242
I20260812 06:59:25.641428 20323 tablet_service.cc:1514] Processing DeleteTablet for tablet 01c467373a404494be1d19fb8a08b505 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:25 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:43636
I20260812 06:59:25.642210 19900 tablet_service.cc:1514] Processing DeleteTablet for tablet 01c467373a404494be1d19fb8a08b505 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:25 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:40662
I20260812 06:59:25.642588 20479 tablet_replica.cc:330] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: stopping tablet replica
I20260812 06:59:25.643049 20480 tablet_replica.cc:330] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: stopping tablet replica
I20260812 06:59:25.643218 20479 raft_consensus.cc:2229] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:59:25.643478 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 19760
I20260812 06:59:25.643465 20480 raft_consensus.cc:2229] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:25.644145 20479 raft_consensus.cc:2258] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:25.645241 20480 raft_consensus.cc:2258] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:25.647219 20479 ts_tablet_manager.cc:1905] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:25.652436 20480 ts_tablet_manager.cc:1905] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:25.656333 20479 ts_tablet_manager.cc:1918] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:25.656648 20479 log.cc:1196] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/01c467373a404494be1d19fb8a08b505
I20260812 06:59:25.657366 20479 ts_tablet_manager.cc:1939] T 01c467373a404494be1d19fb8a08b505 P 5983c4830d8e4b91af2dde459d243f21: Deleting consensus metadata
I20260812 06:59:25.659560 19676 catalog_manager.cc:4841] TS 5983c4830d8e4b91af2dde459d243f21 (127.26.226.2:40395): tablet 01c467373a404494be1d19fb8a08b505 (table test-table [id=ab8192528650450097473206dfc93e21]) successfully deleted
I20260812 06:59:25.659822 20480 ts_tablet_manager.cc:1918] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:25.659988 20480 log.cc:1196] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/01c467373a404494be1d19fb8a08b505
I20260812 06:59:25.660696 20480 ts_tablet_manager.cc:1939] T 01c467373a404494be1d19fb8a08b505 P 2d89bbfec43e4f30bd70dd2d6a6c78c7: Deleting consensus metadata
I20260812 06:59:25.662741 19677 catalog_manager.cc:4841] TS 2d89bbfec43e4f30bd70dd2d6a6c78c7 (127.26.226.3:43509): tablet 01c467373a404494be1d19fb8a08b505 (table test-table [id=ab8192528650450097473206dfc93e21]) successfully deleted
I20260812 06:59:25.663715 20481 tablet_replica.cc:330] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: stopping tablet replica
I20260812 06:59:25.664283 20481 raft_consensus.cc:2229] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:25.664881 20481 raft_consensus.cc:2258] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:25.666919 20481 ts_tablet_manager.cc:1905] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:25.669116 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.644369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.26.226.1:43477)
0812 06:59:25.644985 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.644991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.645009 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:59:25.645069 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.645075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.645163 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.645277 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:25.647367 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.648659 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.650791 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.651484 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.651614 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.660738 (+ 9124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:25.660816 (+ 78us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:25.660846 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:25.660907 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:25.665400 (+ 4493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:25.666459 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:25.666468 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:25.666471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:25.666663 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:25.667134 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:25.667139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:25.667142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:25.667663 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:25.668031 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.668375 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.668631 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484}
I20260812 06:59:25.670174 19998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:25.788938 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 19760
W20260812 06:59:25.815863 19678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.226.1:43477 (error 108)
I20260812 06:59:25.816942 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 20014
I20260812 06:59:25.833164 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.832698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.0.0.1:40678)
0812 06:59:25.832875 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.833010 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 06:59:25.833698 19678 catalog_manager.cc:4584] TS a89275d8057949eb9c552e054bd53c79 (127.26.226.1:43477): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 01c467373a404494be1d19fb8a08b505: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43477: connect: Connection refused (error 111)
I20260812 06:59:25.841063 20433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.818876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.26.226.2:40395)
0812 06:59:25.819113 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.819121 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.819150 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:59:25.819229 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.819237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.819348 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.819475 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:25.822164 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.823752 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.826749 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.827848 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.828018 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.833514 (+ 5496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:25.833536 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:25.833561 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:25.833624 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:25.837657 (+ 4033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:25.838488 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:25.838506 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:25.838509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:25.838660 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:25.839124 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:25.839129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:25.839134 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:25.839640 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:25.839977 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.840427 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.840694 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 06:59:25.842047 20208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:25.887777 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.887155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.0.0.1:40694)
0812 06:59:25.887431 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.887582 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260812 06:59:25.889017 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.888600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.0.0.1:40696)
0812 06:59:25.888771 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.888870 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 06:59:25.949970 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.949394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.0.0.1:40702)
0812 06:59:25.949662 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.949769 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 06:59:25.951166 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.950769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.0.0.1:40714)
0812 06:59:25.950936 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:25.951015 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 06:59:25.960338 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 20014
I20260812 06:59:25.992491 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 20226
I20260812 06:59:26.007001 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.993321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.26.226.3:43509)
0812 06:59:25.993519 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.993523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.993543 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:59:25.993736 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.993741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.993836 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.993932 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:25.995414 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.996244 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.997604 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.998029 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.998144 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:26.001159 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:26.001175 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:26.001192 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:26.001225 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:26.004200 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:26.004956 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:26.004963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:26.004966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:26.005129 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:26.005549 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:26.005554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:26.005556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:26.005957 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:26.006213 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:26.006536 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 06:59:26.006729 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:59:26.007863 20412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:26.031718 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:26.031218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.0.0.1:40728)
0812 06:59:26.031442 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:26.031530 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 06:59:26.032922 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:26.032502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.0.0.1:40736)
0812 06:59:26.032725 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:26.032808 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260812 06:59:26.098812 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 20226
I20260812 06:59:26.127731 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 19650
I20260812 06:59:26.128981 19752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:26.244983 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 19650
I20260812 06:59:26.245019 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:26.244248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:43477 (local address 127.0.0.1:40746)
0812 06:59:26.244659 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:26.244780 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:43477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
I20260812 06:59:26.277635 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34519
--webserver_interface=127.26.226.62
--webserver_port=43207
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:34519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:26.652019 20490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:26.652485 20490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:26.652655 20490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:26.660225 20490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:26.660378 20490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:26.660522 20490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:26.660648 20490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:26.660737 20490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:26.671090 20490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:34519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=43207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:26.672649 20490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:26.674075 20490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:26.687434 20500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:26.687389 20501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:26.687434 20503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:26.688269 20490 server_base.cc:1034] running on GCE node
I20260812 06:59:26.689201 20490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:26.690815 20490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:26.692109 20490 hybrid_clock.cc:648] HybridClock initialized: now 1786517966692009 us; error 89 us; skew 500 ppm
I20260812 06:59:26.692523 20490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:26 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517966, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:26.695848 20490 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:59:26.697620 20490 webserver.cc:460] Webserver started at http://127.26.226.62:43207/ using document root <none> and password file <none>
I20260812 06:59:26.698340 20490 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:26.698452 20490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:26.704396 20490 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:59:26.707535 20515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:26.709162 20490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:26.709323 20490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "42d525d1eb154a16ae71ced739fea360"
format_stamp: "Formatted at 2026-08-12 06:59:20 on dist-test-slave-t3q3"
server_key: "8b7a27faa52ebc95fc205c5596058ac5"
server_key_iv: "b72264e01947e8c04a91847900f05367"
server_key_version: "encryptionkey@0"
I20260812 06:59:26.710331 20490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:26.735414 20490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:26.740748 20490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:26.741019 20490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:26.762710 20490 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:34519
I20260812 06:59:26.762717 20599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:34519 every 8 connection(s)
I20260812 06:59:26.764592 20490 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:26.769439 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 20490
I20260812 06:59:26.770360 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:43477
--local_ip_for_outbound_sockets=127.26.226.1
--tserver_master_addrs=127.26.226.62:34519
--webserver_port=38529
--webserver_interface=127.26.226.1
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:26.772145 20600 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:59:26.776177 20600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Bootstrap starting.
I20260812 06:59:26.798461 20600 log.cc:824] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:26.816697 20600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:59:26.817338 20600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360: Bootstrap complete.
I20260812 06:59:26.827064 20600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:26.828051 20600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42d525d1eb154a16ae71ced739fea360, State: Initialized, Role: FOLLOWER
I20260812 06:59:26.828811 20600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:26.829071 20600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:26.829216 20600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:26.829414 20600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:59:26.833498 20600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:26.834049 20600 leader_election.cc:302] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42d525d1eb154a16ae71ced739fea360; no voters:
I20260812 06:59:26.834779 20600 leader_election.cc:288] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 06:59:26.834918 20606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:59:26.836270 20606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [term 3 LEADER]: Becoming Leader. State: Replica: 42d525d1eb154a16ae71ced739fea360, State: Running, Role: LEADER
I20260812 06:59:26.836855 20606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } }
I20260812 06:59:26.837182 20600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:26.839491 20614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42d525d1eb154a16ae71ced739fea360. Latest consensus state: current_term: 3 leader_uuid: "42d525d1eb154a16ae71ced739fea360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } } }
I20260812 06:59:26.839923 20614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:26.840549 20607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "42d525d1eb154a16ae71ced739fea360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d525d1eb154a16ae71ced739fea360" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34519 } } }
I20260812 06:59:26.840865 20607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d525d1eb154a16ae71ced739fea360 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:26.854085 20624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:26.863492 20624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:26.865553 20624 catalog_manager.cc:1259] Loaded cluster ID: 7c1a1b584a6447ca933aaab8c40e3ea3
I20260812 06:59:26.865679 20624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 06:59:26.868623 20624 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8a668759ad google::LogMessage::Fail() at ??:0
@ 0x7f8a6687888a google::LogMessage::SendToLog() at ??:0
@ 0x7f8a668754de google::LogMessage::Flush() at ??:0
@ 0x7f8a66876a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8a7c2a8512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f8a7c2a5976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8a7c32d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f8a7c32d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8a694c1345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8a694c491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f8a694c4799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8a6949feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8a6c7796db start_thread at ??:0
@ 0x7f8a6570f61f clone at ??:0
W20260812 06:59:27.179215 20604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:27.179498 20604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:27.179597 20604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:27.187095 20604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:27.187216 20604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:27.187337 20604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:59:27.197363 20604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:43477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=38529
--tserver_master_addrs=127.26.226.62:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:27.198629 20604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:27.199900 20604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:27.213487 20668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:27.213568 20669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:27.213595 20673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:27.217621 20604 server_base.cc:1034] running on GCE node
I20260812 06:59:27.218320 20604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:27.219331 20604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:27.220595 20604 hybrid_clock.cc:648] HybridClock initialized: now 1786517967220544 us; error 57 us; skew 500 ppm
I20260812 06:59:27.220957 20604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:27 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517967, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:27.224728 20604 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:59:27.226266 20604 webserver.cc:460] Webserver started at http://127.26.226.1:38529/ using document root <none> and password file <none>
I20260812 06:59:27.227030 20604 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:27.227145 20604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:27.233125 20604 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:59:27.236431 20683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:27.237824 20604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:59:27.237967 20604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "a89275d8057949eb9c552e054bd53c79"
format_stamp: "Formatted at 2026-08-12 06:59:21 on dist-test-slave-t3q3"
server_key: "44e3ea896e40631072f3ee570f6a77b7"
server_key_iv: "c23ce8e53efc425f4f8928ac74c01921"
server_key_version: "encryptionkey@0"
I20260812 06:59:27.238942 20604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:27.265210 20604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:27.269930 20604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:27.270188 20604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:27.271242 20604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:27.273276 20694 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 06:59:27.279129 20694 ts_tablet_manager.cc:1852] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 06:59:27.280220 20694 ts_tablet_manager.cc:1905] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:27.287930 20694 ts_tablet_manager.cc:1918] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:27.288280 20694 log.cc:1196] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/01c467373a404494be1d19fb8a08b505
I20260812 06:59:27.288856 20694 ts_tablet_manager.cc:1939] T 01c467373a404494be1d19fb8a08b505 P a89275d8057949eb9c552e054bd53c79: Deleting consensus metadata
I20260812 06:59:27.289268 20604 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 06:59:27.289407 20604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.000s sys 0.001s
I20260812 06:59:27.289572 20604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:27.289664 20604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:27.325593 20604 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:43477
I20260812 06:59:27.325651 20871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:43477 every 8 connection(s)
I20260812 06:59:27.327174 20604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:27.327906 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 20604
I20260812 06:59:27.328851 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:40395
--local_ip_for_outbound_sockets=127.26.226.2
--tserver_master_addrs=127.26.226.62:34519
--webserver_port=39435
--webserver_interface=127.26.226.2
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:27.333344 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:27.330732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:46491)
0812 06:59:27.331721 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:27.332037 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":403,"threads_started":1}
W20260812 06:59:27.334179 20872 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:34519 (0 consecutive failures): Network error: Failed to ping master at 127.26.226.62:34519: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
I20260812 06:59:27.335209 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:27.334756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:50123)
0812 06:59:27.334953 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:27.335053 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 06:59:27.336308 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:27.335826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:55957)
0812 06:59:27.336006 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:27.336139 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:59:27.336591 20872 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:59:27.685323 20875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:27.685635 20875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:27.685807 20875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:27.693408 20875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:27.693537 20875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:27.693739 20875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:59:27.703737 20875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:40395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=39435
--tserver_master_addrs=127.26.226.62:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:27.705166 20875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:27.706573 20875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:27.720731 20883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:27.720852 20882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:27.720767 20886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:27.721745 20875 server_base.cc:1034] running on GCE node
I20260812 06:59:27.722523 20875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:27.723569 20875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:27.724812 20875 hybrid_clock.cc:648] HybridClock initialized: now 1786517967724768 us; error 65 us; skew 500 ppm
I20260812 06:59:27.725168 20875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:27 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517967, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:27.728806 20875 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:59:27.730479 20875 webserver.cc:460] Webserver started at http://127.26.226.2:39435/ using document root <none> and password file <none>
I20260812 06:59:27.731240 20875 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:27.731405 20875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:27.737370 20875 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:59:27.740414 20896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:27.741859 20875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:59:27.742061 20875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "5983c4830d8e4b91af2dde459d243f21"
format_stamp: "Formatted at 2026-08-12 06:59:21 on dist-test-slave-t3q3"
server_key: "bc2bc8287b54a2917ae279d4e5623b4e"
server_key_iv: "a30af246a7d1df448779c6129f9daa04"
server_key_version: "encryptionkey@0"
I20260812 06:59:27.743265 20875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:27.762085 20875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:27.766726 20875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:27.766970 20875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:27.767941 20875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:27.769362 20875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:27.769456 20875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:27.769542 20875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:27.769604 20875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:27.803524 20875 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:40395
I20260812 06:59:27.803586 21060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:40395 every 8 connection(s)
I20260812 06:59:27.805158 20875 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:27.810456 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:27.808357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:44067)
0812 06:59:27.809250 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:27.809527 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":384,"threads_started":1}
W20260812 06:59:27.811209 21062 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:34519 (0 consecutive failures): Network error: Failed to ping master at 127.26.226.62:34519: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
I20260812 06:59:27.811695 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 20875
I20260812 06:59:27.812103 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:27.811707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:58755)
0812 06:59:27.811874 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:27.811952 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 06:59:27.812522 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:43509
--local_ip_for_outbound_sockets=127.26.226.3
--tserver_master_addrs=127.26.226.62:34519
--webserver_port=45585
--webserver_interface=127.26.226.3
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:27.813119 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:27.812766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:49989)
0812 06:59:27.812913 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:27.813003 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 06:59:27.813338 21062 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:59:28.165333 21065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:28.165647 21065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:28.165832 21065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:28.173309 21065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:28.173431 21065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:28.173655 21065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:59:28.183692 21065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:43509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=45585
--tserver_master_addrs=127.26.226.62:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:28.185111 21065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:28.186491 21065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:28.200759 21074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:28.200860 21079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:28.200826 21076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:28.201768 21065 server_base.cc:1034] running on GCE node
I20260812 06:59:28.202431 21065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:28.203438 21065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:28.204699 21065 hybrid_clock.cc:648] HybridClock initialized: now 1786517968204655 us; error 49 us; skew 500 ppm
I20260812 06:59:28.205053 21065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:28 dist-test-slave-t3q3 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517968, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:28.208503 21065 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:59:28.210161 21065 webserver.cc:460] Webserver started at http://127.26.226.3:45585/ using document root <none> and password file <none>
I20260812 06:59:28.210906 21065 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:28.211072 21065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:28.216826 21065 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:59:28.219956 21092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:28.221550 21065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:59:28.221773 21065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "2d89bbfec43e4f30bd70dd2d6a6c78c7"
format_stamp: "Formatted at 2026-08-12 06:59:22 on dist-test-slave-t3q3"
server_key: "31f796a6f63903e76dae46e51df1c5be"
server_key_iv: "f0beb68f6450b4a08d16cf8b9b2604cf"
server_key_version: "encryptionkey@0"
I20260812 06:59:28.222836 21065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:28.258306 21065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:28.262807 21065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:28.263132 21065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:28.264374 21065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:28.265831 21065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:28.265965 21065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:28.266124 21065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:28.266217 21065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:28.301688 21065 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:43509
I20260812 06:59:28.301733 21275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:43509 every 8 connection(s)
I20260812 06:59:28.303268 21065 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:28.305418 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 21065
I20260812 06:59:28.308588 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:28.306560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:39947)
0812 06:59:28.307454 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:28.307684 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":377,"threads_started":1}
W20260812 06:59:28.309343 21277 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:34519 (0 consecutive failures): Network error: Failed to ping master at 127.26.226.62:34519: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
I20260812 06:59:28.310180 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:28.309877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:53473)
0812 06:59:28.310013 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:28.310079 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 06:59:28.310999 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:28.310711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:35625)
0812 06:59:28.310827 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:28.310896 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 06:59:28.311245 21277 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 06:59:28.337940 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:28.337158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:38413)
0812 06:59:28.337618 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:28.337708 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":213,"threads_started":1}
I20260812 06:59:28.814946 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:28.814076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:57095)
0812 06:59:28.814569 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:28.814694 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":240,"threads_started":1}
I20260812 06:59:29.312719 21287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:29.311804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:58049)
0812 06:59:29.312355 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:29.312458 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":313,"threads_started":1}
I20260812 06:59:29.339581 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:29.338886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:44517)
0812 06:59:29.339320 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:29.339409 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":233,"threads_started":1}
I20260812 06:59:29.816875 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:29.815998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:46985)
0812 06:59:29.816567 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:29.816661 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":314,"threads_started":1}
I20260812 06:59:30.314572 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:30.313752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:40537)
0812 06:59:30.314294 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:30.314385 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":280,"threads_started":1}
I20260812 06:59:30.341611 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:30.340729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:48037)
0812 06:59:30.341317 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:30.341426 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":303,"threads_started":1}
I20260812 06:59:30.818547 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:30.817823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:60517)
0812 06:59:30.818283 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:30.818366 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":240,"threads_started":1}
I20260812 06:59:31.316497 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:31.315477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:39639)
0812 06:59:31.316029 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:31.316296 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":313,"threads_started":1}
I20260812 06:59:31.343487 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:31.342662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:46815)
0812 06:59:31.343192 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:31.343316 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":246,"threads_started":1}
I20260812 06:59:31.820345 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:31.819510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:37243)
0812 06:59:31.819982 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:31.820142 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":228,"threads_started":1}
I20260812 06:59:32.318426 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:32.317592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:49371)
0812 06:59:32.318130 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:32.318229 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":280,"threads_started":1}
I20260812 06:59:32.345283 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:32.344427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:57037)
0812 06:59:32.345009 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:32.345105 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":334,"threads_started":1}
I20260812 06:59:32.822039 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:32.821282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:46505)
0812 06:59:32.821779 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:32.821865 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":248,"threads_started":1}
I20260812 06:59:33.320494 21303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:33.319512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:54421)
0812 06:59:33.320216 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:33.320313 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":363,"threads_started":1}
I20260812 06:59:33.346992 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:33.346228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:38199)
0812 06:59:33.346666 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:33.346753 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":223,"threads_started":1}
I20260812 06:59:33.824280 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:33.823192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:48191)
0812 06:59:33.823805 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:33.823937 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":311,"threads_started":1}
I20260812 06:59:34.323256 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:34.321762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:58633)
0812 06:59:34.322638 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:34.322788 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":460,"threads_started":1}
I20260812 06:59:34.350311 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:34.348532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:33175)
0812 06:59:34.349588 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:34.349816 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":874,"thread_start_us":510,"threads_started":1}
I20260812 06:59:34.827245 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:34.825645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:37257)
0812 06:59:34.826707 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:34.826860 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":405,"threads_started":1}
I20260812 06:59:35.326550 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:35.324769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:40717)
0812 06:59:35.325741 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:35.326027 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":455,"threads_started":1}
I20260812 06:59:35.353317 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:35.351824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:43027)
0812 06:59:35.352911 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:35.353055 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":670,"threads_started":1}
I20260812 06:59:35.829227 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:35.828446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:54309)
0812 06:59:35.828946 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:35.829035 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":256,"threads_started":1}
I20260812 06:59:36.328707 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:36.327965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:49955)
0812 06:59:36.328468 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:36.328553 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":290,"threads_started":1}
I20260812 06:59:36.355301 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:36.354429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:57979)
0812 06:59:36.354971 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:36.355074 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":281,"threads_started":1}
I20260812 06:59:36.831032 21318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:36.830291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:40779)
0812 06:59:36.830772 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:36.830858 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":227,"threads_started":1}
I20260812 06:59:37.330374 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:37.329579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:60679)
0812 06:59:37.330097 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:37.330213 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":239,"threads_started":1}
I20260812 06:59:37.357049 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:37.356356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:43609)
0812 06:59:37.356800 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:37.356885 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":232,"threads_started":1}
I20260812 06:59:37.832695 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:37.831936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:38699)
0812 06:59:37.832425 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:37.832511 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":295,"threads_started":1}
I20260812 06:59:38.332352 21326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:38.331435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:42669)
0812 06:59:38.332048 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:38.332185 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":346,"threads_started":1}
I20260812 06:59:38.358816 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:38.358042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:47677)
0812 06:59:38.358507 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:38.358605 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":256,"threads_started":1}
I20260812 06:59:38.834499 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:38.833648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:38191)
0812 06:59:38.834174 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:38.834302 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":282,"threads_started":1}
I20260812 06:59:39.334321 21330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:39.333519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:41247)
0812 06:59:39.334019 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:39.334139 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":276,"threads_started":1}
I20260812 06:59:39.360414 21331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:39.359691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:57633)
0812 06:59:39.360164 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:39.360249 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":218,"threads_started":1}
I20260812 06:59:39.836256 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:39.835478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:49149)
0812 06:59:39.835931 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:39.836020 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":229,"threads_started":1}
I20260812 06:59:40.335947 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:40.335215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:50195)
0812 06:59:40.335710 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:40.335797 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":276,"threads_started":1}
I20260812 06:59:40.361984 21336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:40.361252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:45969)
0812 06:59:40.361693 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:40.361796 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":225,"threads_started":1}
I20260812 06:59:40.837884 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:40.837124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:45383)
0812 06:59:40.837593 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:40.837692 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":220,"threads_started":1}
I20260812 06:59:41.337875 21341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:41.336920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:35437)
0812 06:59:41.337481 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:41.337626 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":256,"threads_started":1}
I20260812 06:59:41.363612 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:41.362890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:37573)
0812 06:59:41.363334 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:41.363419 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":223,"threads_started":1}
I20260812 06:59:41.839715 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:41.838932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:60661)
0812 06:59:41.839411 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:41.839514 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":244,"threads_started":1}
I20260812 06:59:42.339565 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:42.338809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:45167)
0812 06:59:42.339278 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:42.339363 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":251,"threads_started":1}
I20260812 06:59:42.365795 21352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:42.364956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.1:37865)
0812 06:59:42.365459 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:42.365598 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":242,"threads_started":1}
I20260812 06:59:42.842003 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:42.841164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.2:33493)
0812 06:59:42.841720 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:42.841812 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":307,"threads_started":1}
I20260812 06:59:43.314787 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 20604
I20260812 06:59:43.338425 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:43.316304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57786 (local address 127.26.226.1:43477)
0812 06:59:43.317093 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:43.317107 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:43.317146 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:59:43.318856 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:43.318867 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:43.319298 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:43.319615 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:43.321212 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:43.322458 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:43.323725 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:43.324542 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:43.324687 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:43.331907 (+ 7220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:43.331915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:43.331932 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:43.331972 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:43.334888 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:43.335626 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:43.335634 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:43.335641 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:43.335816 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:43.336300 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:43.336304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:43.336306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:43.336847 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:43.337199 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:43.337691 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 06:59:43.338128 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":473,"threads_started":1}
I20260812 06:59:43.339888 20864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:43.341866 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:43.340904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34519 (local address 127.26.226.3:60689)
0812 06:59:43.341508 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:43.341627 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:34519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":300,"threads_started":1}
I20260812 06:59:43.425503 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 20604
I20260812 06:59:43.460996 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 20875
I20260812 06:59:43.480414 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:43.462090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37798 (local address 127.26.226.2:40395)
0812 06:59:43.462652 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:43.462667 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:43.462707 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:59:43.464347 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:43.464356 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:43.464779 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:43.465072 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:43.466488 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:43.467774 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:43.469385 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:43.470466 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:43.470692 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:43.473945 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:43.473954 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:43.473970 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:43.474014 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:43.477134 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:43.477821 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:43.477827 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:43.477834 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:43.478007 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:43.478360 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:43.478363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:43.478365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:43.478876 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:43.479115 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:43.479558 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 06:59:43.479998 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":271,"threads_started":1}
I20260812 06:59:43.481839 21050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:43.565500 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 20875
I20260812 06:59:43.593333 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 21065
I20260812 06:59:43.612807 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:43.594383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.26.226.3:43509)
0812 06:59:43.594965 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:43.594980 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:43.595026 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:59:43.596766 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:43.596777 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:43.597225 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:43.597603 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:43.599216 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:43.600561 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:43.601883 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:43.602575 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:43.602787 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:43.605902 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:43.605910 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:43.605927 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:43.605973 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:43.609015 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:43.609790 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:43.609798 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:43.609805 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:43.609987 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:43.610433 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:43.610438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:43.610443 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:43.611012 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:43.611363 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:43.611834 (+ 471us) server_negotiation.cc:299] Negotiation successful
0812 06:59:43.612481 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":267,"threads_started":1}
I20260812 06:59:43.614300 21270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:43.698899 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 21065
2026-08-12T06:59:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25287 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:43 dist-test-slave-t3q3 krb5kdc[21380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:43 dist-test-slave-t3q3 krb5kdc[21380](info): set up 2 sockets
Aug 12 06:59:43 dist-test-slave-t3q3 krb5kdc[21380](info): commencing operation
krb5kdc: starting...
W20260812 06:59:45.065294 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.006s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 06:59:45 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 06:59:45.381826 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:43081
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:43081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:45.734308 21401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:45.734598 21401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:45.734692 21401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:45.742169 21401 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 06:59:45.742280 21401 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 06:59:45.742319 21401 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 06:59:45.742365 21401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:45.742417 21401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:45.742487 21401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:45.742534 21401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:45.742589 21401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:45.752947 21401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:43081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:43081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:45.754280 21401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:45.755649 21401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:45.768123 21406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:45.768093 21409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:45.768436 21401 server_base.cc:1034] running on GCE node
W20260812 06:59:45.768047 21412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:45.769593 21401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:45.771106 21401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:45.772351 21401 hybrid_clock.cc:648] HybridClock initialized: now 1786517985772273 us; error 62 us; skew 500 ppm
I20260812 06:59:45.772704 21401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:45 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517985, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:45.776038 21401 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 06:59:45.777563 21401 webserver.cc:460] Webserver started at http://127.26.226.62:37707/ using document root <none> and password file <none>
I20260812 06:59:45.778247 21401 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:45.778335 21401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:45.778628 21401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:45.781702 21401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "32cd4d0f3b224ca0bd37eafc673fabce"
format_stamp: "Formatted at 2026-08-12 06:59:45 on dist-test-slave-t3q3"
server_key: "9fd59896015ead8084ef3493d0d11046"
server_key_iv: "1e168bc14164361488d6c847f98cf07d"
server_key_version: "encryptionkey@0"
I20260812 06:59:45.782449 21401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "32cd4d0f3b224ca0bd37eafc673fabce"
format_stamp: "Formatted at 2026-08-12 06:59:45 on dist-test-slave-t3q3"
server_key: "9fd59896015ead8084ef3493d0d11046"
server_key_iv: "1e168bc14164361488d6c847f98cf07d"
server_key_version: "encryptionkey@0"
I20260812 06:59:45.787744 21401 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 06:59:45.791285 21421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:45.792694 21401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:45.792840 21401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "32cd4d0f3b224ca0bd37eafc673fabce"
format_stamp: "Formatted at 2026-08-12 06:59:45 on dist-test-slave-t3q3"
server_key: "9fd59896015ead8084ef3493d0d11046"
server_key_iv: "1e168bc14164361488d6c847f98cf07d"
server_key_version: "encryptionkey@0"
I20260812 06:59:45.793036 21401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:45.806041 21401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:45.810343 21401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:45.810564 21401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:45.829445 21401 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:43081
I20260812 06:59:45.829465 21504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:43081 every 8 connection(s)
I20260812 06:59:45.830821 21401 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:45.832413 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 21401
I20260812 06:59:45.832635 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 06:59:45.833576 27528 external_mini_cluster.cc:1428] Setting key b5ffb2bc2b7487aaaec51eb9fafb3a6c
I20260812 06:59:45.835826 21505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:45.847280 21505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Bootstrap starting.
Aug 12 06:59:45 dist-test-slave-t3q3 krb5kdc[21380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:45.850626 21505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:45.851770 21505 log.cc:824] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:45.854604 21505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: No bootstrap required, opened a new log
I20260812 06:59:45.856441 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:45.835872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.26.226.62:43081)
0812 06:59:45.836685 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:45.836697 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:45.836737 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:59:45.838557 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:45.838580 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:45.838967 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:45.839385 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:45.841045 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:45.842338 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:45.843716 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:45.844496 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:45.844712 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:45.849145 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:45.849153 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:45.849168 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:45.849203 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:45.852373 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:45.853100 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:45.853108 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:45.853129 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:45.853301 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:45.853699 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:45.853705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:45.853708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:45.854218 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:45.854512 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:45.854987 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 06:59:45.855421 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":372,"threads_started":1}
I20260812 06:59:45.862437 21505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 06:59:45.862824 21505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:45.862934 21505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32cd4d0f3b224ca0bd37eafc673fabce, State: Initialized, Role: FOLLOWER
I20260812 06:59:45.863513 21505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 06:59:45.863766 21505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:45.863915 21505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:45.864135 21505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:45.866701 21505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 06:59:45.867189 21505 leader_election.cc:302] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32cd4d0f3b224ca0bd37eafc673fabce; no voters:
I20260812 06:59:45.867818 21505 leader_election.cc:288] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:45.867900 21517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:45.868656 21517 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 1 LEADER]: Becoming Leader. State: Replica: 32cd4d0f3b224ca0bd37eafc673fabce, State: Running, Role: LEADER
I20260812 06:59:45.869196 21517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 06:59:45.870131 21505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:45.872336 21518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } } }
I20260812 06:59:45.872404 21519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32cd4d0f3b224ca0bd37eafc673fabce. Latest consensus state: current_term: 1 leader_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } } }
I20260812 06:59:45.872816 21518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:45.872885 21519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:45.875809 21525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:45.881201 21525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:45.889750 21525 catalog_manager.cc:1347] Generated new cluster ID: e4b0fa658d2b4833b00946a444fa394d
I20260812 06:59:45.889868 21525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:45.910696 21525 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:45.912200 21525 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:45.922279 21525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Generated new TSK 0
I20260812 06:59:45.923123 21525 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 06:59:46.000670 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43081
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:46.355868 21548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:46.356235 21548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:46.356434 21548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:46.363948 21548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:46.364120 21548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:46.364332 21548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 06:59:46.374544 21548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:46.376101 21548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:46.377588 21548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:46.390877 21553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:46.390925 21555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:46.390951 21558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:46.391891 21548 server_base.cc:1034] running on GCE node
I20260812 06:59:46.392681 21548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:46.393699 21548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:46.394933 21548 hybrid_clock.cc:648] HybridClock initialized: now 1786517986394888 us; error 28 us; skew 500 ppm
I20260812 06:59:46.395285 21548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:46 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517986, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:46.398787 21548 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 06:59:46.400334 21548 webserver.cc:460] Webserver started at http://127.26.226.1:33431/ using document root <none> and password file <none>
I20260812 06:59:46.401003 21548 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:46.401118 21548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:46.401391 21548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:46.404464 21548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "decc879a4420405a955a509ba6b00cb9"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-t3q3"
server_key: "87756bbef3d8fc1c5e7f70267696c37f"
server_key_iv: "c0dd9dabca97ddf53639439d5c4d87d5"
server_key_version: "encryptionkey@0"
I20260812 06:59:46.405200 21548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "decc879a4420405a955a509ba6b00cb9"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-t3q3"
server_key: "87756bbef3d8fc1c5e7f70267696c37f"
server_key_iv: "c0dd9dabca97ddf53639439d5c4d87d5"
server_key_version: "encryptionkey@0"
I20260812 06:59:46.410449 21548 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:59:46.414391 21569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:46.415721 21548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:59:46.415897 21548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "decc879a4420405a955a509ba6b00cb9"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-t3q3"
server_key: "87756bbef3d8fc1c5e7f70267696c37f"
server_key_iv: "c0dd9dabca97ddf53639439d5c4d87d5"
server_key_version: "encryptionkey@0"
I20260812 06:59:46.416149 21548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:46.434558 21548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:46.438599 21548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:46.438895 21548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:46.439947 21548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:46.441344 21548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:46.441452 21548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:46.441612 21548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:46.441689 21548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:46.476958 21548 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:40821
I20260812 06:59:46.477005 21761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:40821 every 8 connection(s)
I20260812 06:59:46.478264 21548 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:46.483584 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 21548
I20260812 06:59:46.483814 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 06:59:46.484583 27528 external_mini_cluster.cc:1428] Setting key ad5f4194d9f2d63674555a0c5cbce955
Aug 12 06:59:46 dist-test-slave-t3q3 krb5kdc[21380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517986, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:46.501605 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:46.481238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:42993 (local address 127.26.226.62:43081)
0812 06:59:46.481911 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:46.481917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:46.483011 (+ 1094us) server_negotiation.cc:407] Connection header received
0812 06:59:46.485182 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:46.485188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:46.485286 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:46.485425 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:46.487284 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:46.488475 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:46.490027 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:46.490727 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:46.490905 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:46.494600 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:46.494609 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:46.494618 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:46.494663 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:46.498475 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:46.499161 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:46.499169 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:46.499175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:46.499327 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:46.499735 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:46.499742 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:46.499748 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:46.500153 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:46.500304 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:46.500983 (+ 679us) server_negotiation.cc:299] Negotiation successful
0812 06:59:46.501214 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":288,"threads_started":1}
I20260812 06:59:46.502574 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:46.481840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:42993)
0812 06:59:46.482823 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:46.482882 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:46.484908 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:46.485660 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:46.485671 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:46.486444 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:46.487101 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:46.487120 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:46.488643 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:46.488647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:46.489874 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:46.489886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:46.490948 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:46.490952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:46.491158 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:46.491865 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:46.491893 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:46.494360 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:46.498613 (+ 4253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:46.498620 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:46.498632 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:46.499037 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:46.499437 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:46.499440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:46.499443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:46.499627 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:46.500287 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:46.500293 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:46.500606 (+ 313us) client_negotiation.cc:769] Sending connection context
0812 06:59:46.500886 (+ 280us) client_negotiation.cc:240] Negotiation successful
0812 06:59:46.501423 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":408,"threads_started":1}
I20260812 06:59:46.504114 21762 heartbeater.cc:347] Connected to a master server at 127.26.226.62:43081
I20260812 06:59:46.504433 21762 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:46.505160 21762 heartbeater.cc:511] Master 127.26.226.62:43081 requested a full tablet report, sending...
I20260812 06:59:46.507328 21449 ts_manager.cc:194] Registered new tserver with Master: decc879a4420405a955a509ba6b00cb9 (127.26.226.1:40821)
I20260812 06:59:46.508852 21449 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:42993
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 06:59:46.555891 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43081
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:46.933652 21769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:46.934015 21769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:46.934222 21769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:46.941684 21769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:46.941802 21769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:46.942014 21769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 06:59:46.951941 21769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:46.953358 21769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:46.954741 21769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:46.968449 21775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:46.968535 21777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:46.968484 21781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:46.969552 21769 server_base.cc:1034] running on GCE node
I20260812 06:59:46.970180 21769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:46.971186 21769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:46.972389 21769 hybrid_clock.cc:648] HybridClock initialized: now 1786517986972320 us; error 58 us; skew 500 ppm
I20260812 06:59:46.972685 21769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:46 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517986, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:46.976156 21769 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 06:59:46.977768 21769 webserver.cc:460] Webserver started at http://127.26.226.2:37251/ using document root <none> and password file <none>
I20260812 06:59:46.978507 21769 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:46.978659 21769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:46.979003 21769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:46.982121 21769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-t3q3"
server_key: "ff3d591753afba4e845a16ab4326cb9e"
server_key_iv: "f2d42abd338a6fc7b327f8c4957c43f6"
server_key_version: "encryptionkey@0"
I20260812 06:59:46.982920 21769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-t3q3"
server_key: "ff3d591753afba4e845a16ab4326cb9e"
server_key_iv: "f2d42abd338a6fc7b327f8c4957c43f6"
server_key_version: "encryptionkey@0"
I20260812 06:59:46.988247 21769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 06:59:46.991766 21792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:46.993217 21769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:46.993399 21769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-t3q3"
server_key: "ff3d591753afba4e845a16ab4326cb9e"
server_key_iv: "f2d42abd338a6fc7b327f8c4957c43f6"
server_key_version: "encryptionkey@0"
I20260812 06:59:46.993615 21769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:47.014310 21769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:47.018407 21769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:47.018735 21769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:47.019840 21769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:47.021361 21769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:47.021468 21769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:47.021627 21769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:47.021700 21769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:47.057272 21769 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33433
I20260812 06:59:47.057332 21976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33433 every 8 connection(s)
I20260812 06:59:47.058559 21769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:47.061194 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 21769
I20260812 06:59:47.061400 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 06:59:47.061954 27528 external_mini_cluster.cc:1428] Setting key d517733d79859064ae703c81690ce1b4
Aug 12 06:59:47 dist-test-slave-t3q3 krb5kdc[21380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517986, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:47.081094 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.061238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:51915 (local address 127.26.226.62:43081)
0812 06:59:47.061770 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:47.061777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:47.062868 (+ 1091us) server_negotiation.cc:407] Connection header received
0812 06:59:47.064991 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:47.064999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:47.065098 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:47.065215 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:47.066725 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.067895 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.069467 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.070207 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.070356 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.074200 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:47.074210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:47.074218 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:47.074264 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:47.077732 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:47.078699 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:47.078703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:47.078705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:47.078826 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:47.079286 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:47.079290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:47.079292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:47.079556 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:47.079734 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:47.080483 (+ 749us) server_negotiation.cc:299] Negotiation successful
0812 06:59:47.080705 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":257,"threads_started":1}
I20260812 06:59:47.082056 21984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.061839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:51915)
0812 06:59:47.062697 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:47.062744 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:47.064700 (+ 1956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:47.065443 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:47.065451 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:47.065987 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:47.066544 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.066562 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.068039 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.068042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.069319 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.069332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.070417 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.070421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.070706 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.071402 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:47.071429 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:47.073960 (+ 2531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:47.077869 (+ 3909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:47.077881 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:47.077899 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:47.078577 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:47.078923 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:47.078926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:47.078928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:47.079135 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:47.079700 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:47.079710 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:47.080026 (+ 316us) client_negotiation.cc:769] Sending connection context
0812 06:59:47.080385 (+ 359us) client_negotiation.cc:240] Negotiation successful
0812 06:59:47.080915 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":353,"threads_started":1}
I20260812 06:59:47.083540 21977 heartbeater.cc:347] Connected to a master server at 127.26.226.62:43081
I20260812 06:59:47.083848 21977 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:47.084621 21977 heartbeater.cc:511] Master 127.26.226.62:43081 requested a full tablet report, sending...
I20260812 06:59:47.086184 21449 ts_manager.cc:194] Registered new tserver with Master: c4f8bc5e74a243bca3e5c8759ea92fbd (127.26.226.2:33433)
I20260812 06:59:47.087433 21449 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:51915
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 06:59:47.131474 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43081
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:47.492338 21988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:47.492625 21988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:47.492723 21988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:47.500379 21988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:47.500497 21988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:47.500623 21988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 06:59:47.510663 21988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:43081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 06:59:47.511643 21762 heartbeater.cc:503] Master 127.26.226.62:43081 was elected leader, sending a full tablet report...
I20260812 06:59:47.512015 21988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:47.513720 21988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:47.527534 22001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:47.527586 21995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:47.527486 21997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:47.528497 21988 server_base.cc:1034] running on GCE node
I20260812 06:59:47.529129 21988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:47.530079 21988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:47.531293 21988 hybrid_clock.cc:648] HybridClock initialized: now 1786517987531249 us; error 33 us; skew 500 ppm
I20260812 06:59:47.531621 21988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:59:47 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517987, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:47.534866 21988 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 06:59:47.536811 21988 webserver.cc:460] Webserver started at http://127.26.226.3:41305/ using document root <none> and password file <none>
I20260812 06:59:47.537521 21988 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:47.537659 21988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:47.537994 21988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:47.541129 21988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f"
format_stamp: "Formatted at 2026-08-12 06:59:47 on dist-test-slave-t3q3"
server_key: "cb72053c3b09ff1515081f93ca1d86e5"
server_key_iv: "73dc76ee489048fa30e8bc69c5c02d94"
server_key_version: "encryptionkey@0"
I20260812 06:59:47.541905 21988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f"
format_stamp: "Formatted at 2026-08-12 06:59:47 on dist-test-slave-t3q3"
server_key: "cb72053c3b09ff1515081f93ca1d86e5"
server_key_iv: "73dc76ee489048fa30e8bc69c5c02d94"
server_key_version: "encryptionkey@0"
I20260812 06:59:47.547254 21988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 06:59:47.550889 22011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:47.552412 21988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:47.552584 21988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f"
format_stamp: "Formatted at 2026-08-12 06:59:47 on dist-test-slave-t3q3"
server_key: "cb72053c3b09ff1515081f93ca1d86e5"
server_key_iv: "73dc76ee489048fa30e8bc69c5c02d94"
server_key_version: "encryptionkey@0"
I20260812 06:59:47.552801 21988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:47.565711 21988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:47.570034 21988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:47.570348 21988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:47.571359 21988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:47.572763 21988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:47.572870 21988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:47.573028 21988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:47.573102 21988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:47.607851 21988 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:44297
I20260812 06:59:47.607893 22209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:44297 every 8 connection(s)
I20260812 06:59:47.609159 21988 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:47.612552 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 21988
I20260812 06:59:47.612785 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 06:59:47.613533 27528 external_mini_cluster.cc:1428] Setting key e1582f161123d53f3f2235b9e037accf
Aug 12 06:59:47 dist-test-slave-t3q3 krb5kdc[21380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517987, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 06:59:47.631170 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.611839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:43243 (local address 127.26.226.62:43081)
0812 06:59:47.612361 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:47.612366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:47.613602 (+ 1236us) server_negotiation.cc:407] Connection header received
0812 06:59:47.615908 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:47.615915 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:47.616030 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:47.616196 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:47.617818 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.618697 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.620397 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.620849 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.620957 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.624525 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:47.624530 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:47.624535 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:47.624574 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:47.627662 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:47.628680 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:47.628685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:47.628687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:47.628861 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:47.629420 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:47.629426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:47.629429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:47.629865 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:47.629993 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:47.630679 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 06:59:47.630850 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":292,"threads_started":1}
I20260812 06:59:47.632189 22214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.612507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:43243)
0812 06:59:47.613433 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:47.613472 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:47.615540 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:47.616391 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:47.616399 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:47.617067 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:47.617654 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.617673 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.618830 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.618833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.620250 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.620268 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.621048 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.621052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.621244 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.621909 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:47.621931 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:47.624312 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:47.627851 (+ 3539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:47.627862 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:47.627881 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:47.628531 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:47.629017 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:47.629022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:47.629025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:47.629291 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:47.629990 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:47.629996 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:47.630340 (+ 344us) client_negotiation.cc:769] Sending connection context
0812 06:59:47.630577 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 06:59:47.630991 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":412,"threads_started":1}
I20260812 06:59:47.633658 22210 heartbeater.cc:347] Connected to a master server at 127.26.226.62:43081
I20260812 06:59:47.633980 22210 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:47.634683 22210 heartbeater.cc:511] Master 127.26.226.62:43081 requested a full tablet report, sending...
I20260812 06:59:47.636209 21449 ts_manager.cc:194] Registered new tserver with Master: a9cf9c12d09c4a0eaaca8d23f31b061f (127.26.226.3:44297)
I20260812 06:59:47.637051 21449 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:43243
I20260812 06:59:47.642153 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:47.659502 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.646861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41160 (local address 127.26.226.62:43081)
0812 06:59:47.647100 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:47.647106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:47.647444 (+ 338us) server_negotiation.cc:407] Connection header received
0812 06:59:47.647580 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:47.647583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:47.647647 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:47.647733 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:47.649270 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.649994 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.651336 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.651745 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.651837 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.654404 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:47.654409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:47.654413 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:47.654442 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:47.657143 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:47.657789 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:47.657793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:47.657795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:47.657918 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:47.658259 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:47.658262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:47.658264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:47.658571 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:47.658754 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:47.659005 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 06:59:47.659180 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:59:47.663599 21449 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:59:47.670717 21449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:59:47.694954 22232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.684439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:40821 (local address 127.0.0.1:51484)
0812 06:59:47.686309 (+ 1870us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:47.686332 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:47.686484 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:47.687381 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:47.687392 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:47.687435 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:47.687961 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.687982 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.689771 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.689777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.692318 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.692338 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.693987 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.693994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.694197 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.694251 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 06:59:47.694394 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 06:59:47.694578 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1715,"thread_start_us":1540,"threads_started":1}
I20260812 06:59:47.695912 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.683910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33433 (local address 127.0.0.1:60458)
0812 06:59:47.684851 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:47.684900 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:47.685081 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:47.688646 (+ 3565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:47.688650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:47.688674 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:47.689039 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.689053 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.690862 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.690870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.693166 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.693182 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.695064 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.695071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.695262 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.695296 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:59:47.695370 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 06:59:47.695508 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":3,"thread_start_us":408,"threads_started":1}
I20260812 06:59:47.696017 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.684940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51484 (local address 127.26.226.1:40821)
0812 06:59:47.685783 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:47.685796 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:47.686647 (+ 851us) server_negotiation.cc:407] Connection header received
0812 06:59:47.686781 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:47.686794 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:47.687103 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:47.687337 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:47.688251 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.689633 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.692498 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.693776 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.693932 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.694172 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:47.694450 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 06:59:47.695684 (+ 1234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":463,"threads_started":1}
I20260812 06:59:47.697055 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.684127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.26.226.2:33433)
0812 06:59:47.688020 (+ 3893us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:47.688032 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:47.688084 (+ 52us) server_negotiation.cc:407] Connection header received
0812 06:59:47.688193 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:47.688200 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:47.688422 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:47.689248 (+ 826us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:47.689477 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.690729 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.693349 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.694792 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.696307 (+ 1515us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.696519 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:47.696572 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 06:59:47.696724 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3717,"thread_start_us":3599,"threads_started":1}
I20260812 06:59:47.697703 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.685250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:44297 (local address 127.0.0.1:57398)
0812 06:59:47.688367 (+ 3117us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:47.688384 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:47.688515 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:47.689470 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:47.689474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:47.689497 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:47.689975 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.689989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.691971 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.691976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.693793 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.693811 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.695852 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.695858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.696052 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.696119 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 06:59:47.697184 (+ 1065us) client_negotiation.cc:240] Negotiation successful
0812 06:59:47.697319 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2999,"mutex_wait_us":1,"thread_start_us":2857,"threads_started":1}
I20260812 06:59:47.698045 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.685980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.26.226.3:44297)
0812 06:59:47.686757 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:47.686768 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:47.688648 (+ 1880us) server_negotiation.cc:407] Connection header received
0812 06:59:47.688761 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:47.688770 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:47.688981 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:47.689196 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:47.690185 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.691763 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.693977 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.695389 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.695498 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.695725 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:47.697298 (+ 1573us) server_negotiation.cc:299] Negotiation successful
0812 06:59:47.697461 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":418,"threads_started":1}
I20260812 06:59:47.700457 21872 tablet_service.cc:1467] Processing CreateTablet for tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab (DEFAULT_TABLE table=test-table [id=8f2d537d37714d058d12a44f4f18bc04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:47.700333 21650 tablet_service.cc:1467] Processing CreateTablet for tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab (DEFAULT_TABLE table=test-table [id=8f2d537d37714d058d12a44f4f18bc04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:47.701234 22093 tablet_service.cc:1467] Processing CreateTablet for tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab (DEFAULT_TABLE table=test-table [id=8f2d537d37714d058d12a44f4f18bc04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:47.701861 21650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:47.701882 21872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:47.702419 22093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:47.713778 22238 tablet_bootstrap.cc:492] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: Bootstrap starting.
I20260812 06:59:47.716837 22239 tablet_bootstrap.cc:492] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: Bootstrap starting.
I20260812 06:59:47.717885 22238 tablet_bootstrap.cc:654] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:47.718274 22241 tablet_bootstrap.cc:492] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: Bootstrap starting.
I20260812 06:59:47.719342 22238 log.cc:824] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:47.721743 22239 tablet_bootstrap.cc:654] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:47.721920 22241 tablet_bootstrap.cc:654] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:47.722909 22241 log.cc:824] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:47.723059 22238 tablet_bootstrap.cc:492] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: No bootstrap required, opened a new log
I20260812 06:59:47.723032 22239 log.cc:824] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:47.723342 22238 ts_tablet_manager.cc:1397] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.002s
I20260812 06:59:47.725862 22239 tablet_bootstrap.cc:492] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: No bootstrap required, opened a new log
I20260812 06:59:47.726042 22241 tablet_bootstrap.cc:492] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: No bootstrap required, opened a new log
I20260812 06:59:47.726207 22239 ts_tablet_manager.cc:1397] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260812 06:59:47.726311 22241 ts_tablet_manager.cc:1397] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 06:59:47.733752 22239 raft_consensus.cc:348] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 06:59:47.734158 22239 raft_consensus.cc:374] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:47.734265 22239 raft_consensus.cc:729] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f8bc5e74a243bca3e5c8759ea92fbd, State: Initialized, Role: FOLLOWER
I20260812 06:59:47.734288 22238 raft_consensus.cc:348] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 06:59:47.734768 22238 raft_consensus.cc:374] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:47.734874 22238 raft_consensus.cc:729] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9cf9c12d09c4a0eaaca8d23f31b061f, State: Initialized, Role: FOLLOWER
I20260812 06:59:47.734961 22239 consensus_queue.cc:260] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 06:59:47.735473 22238 consensus_queue.cc:260] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 06:59:47.736418 21977 heartbeater.cc:503] Master 127.26.226.62:43081 was elected leader, sending a full tablet report...
I20260812 06:59:47.737236 22241 raft_consensus.cc:348] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 06:59:47.737762 22241 raft_consensus.cc:374] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:47.737916 22241 raft_consensus.cc:729] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: decc879a4420405a955a509ba6b00cb9, State: Initialized, Role: FOLLOWER
I20260812 06:59:47.738759 22241 consensus_queue.cc:260] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 06:59:47.739014 22239 ts_tablet_manager.cc:1428] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: Time spent starting tablet: real 0.013s user 0.010s sys 0.000s
I20260812 06:59:47.740106 22210 heartbeater.cc:503] Master 127.26.226.62:43081 was elected leader, sending a full tablet report...
I20260812 06:59:47.741072 22238 ts_tablet_manager.cc:1428] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: Time spent starting tablet: real 0.018s user 0.009s sys 0.008s
I20260812 06:59:47.748185 22250 raft_consensus.cc:482] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:47.749011 22241 ts_tablet_manager.cc:1428] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: Time spent starting tablet: real 0.023s user 0.013s sys 0.008s
I20260812 06:59:47.749325 22250 raft_consensus.cc:504] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 06:59:47.757385 22250 leader_election.cc:288] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4f8bc5e74a243bca3e5c8759ea92fbd (127.26.226.2:33433), a9cf9c12d09c4a0eaaca8d23f31b061f (127.26.226.3:44297)
I20260812 06:59:47.760680 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.752372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:57741 (local address 127.26.226.2:33433)
0812 06:59:47.752593 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:47.752600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:47.752755 (+ 155us) server_negotiation.cc:407] Connection header received
0812 06:59:47.753079 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:47.753088 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:47.753177 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:47.753243 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:47.754201 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.755671 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.758804 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.759594 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.759787 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.759866 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:47.760251 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 06:59:47.760399 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 06:59:47.760890 22252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.751883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33433 (local address 127.26.226.1:57741)
0812 06:59:47.752643 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:47.752664 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:47.752874 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:47.753486 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:47.753493 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:47.753519 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:47.753983 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.754004 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.755892 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.755900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.758636 (+ 2736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.758653 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.759827 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.759834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.760024 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.760048 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:59:47.760224 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 06:59:47.760440 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":54,"thread_start_us":718,"threads_started":1}
I20260812 06:59:47.762003 21901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4b1fe76ac0b4e7a9080c8a6d1e092ab" candidate_uuid: "decc879a4420405a955a509ba6b00cb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" is_pre_election: true
I20260812 06:59:47.762540 21901 raft_consensus.cc:2454] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate decc879a4420405a955a509ba6b00cb9 in term 0.
I20260812 06:59:47.763299 21570 leader_election.cc:302] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4f8bc5e74a243bca3e5c8759ea92fbd, decc879a4420405a955a509ba6b00cb9; no voters:
I20260812 06:59:47.763702 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.753707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:38901 (local address 127.26.226.3:44297)
0812 06:59:47.753924 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:47.753934 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:47.757756 (+ 3822us) server_negotiation.cc:407] Connection header received
0812 06:59:47.758109 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:47.758123 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:47.758203 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:47.758300 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:47.758886 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.759657 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.761854 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.762765 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.762889 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.762985 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:47.763298 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 06:59:47.763432 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 06:59:47.763849 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.753234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:44297 (local address 127.26.226.1:38901)
0812 06:59:47.757593 (+ 4359us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:47.757626 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:47.757947 (+ 321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:47.758323 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:47.758326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:47.758345 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:47.758698 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.758705 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.759812 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.759819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.761666 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:47.761682 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.762988 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.762994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:47.763132 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.763155 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:59:47.763269 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 06:59:47.763404 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4173,"mutex_wait_us":19,"thread_start_us":3925,"threads_started":1}
I20260812 06:59:47.763880 22251 raft_consensus.cc:2781] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:47.764101 22251 raft_consensus.cc:482] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:47.764286 22251 raft_consensus.cc:3037] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:47.764859 22127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4b1fe76ac0b4e7a9080c8a6d1e092ab" candidate_uuid: "decc879a4420405a955a509ba6b00cb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" is_pre_election: true
I20260812 06:59:47.765353 22127 raft_consensus.cc:2454] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate decc879a4420405a955a509ba6b00cb9 in term 0.
I20260812 06:59:47.766963 22251 raft_consensus.cc:504] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 06:59:47.767653 22251 leader_election.cc:288] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [CANDIDATE]: Term 1 election: Requested vote from peers c4f8bc5e74a243bca3e5c8759ea92fbd (127.26.226.2:33433), a9cf9c12d09c4a0eaaca8d23f31b061f (127.26.226.3:44297)
I20260812 06:59:47.768162 21901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4b1fe76ac0b4e7a9080c8a6d1e092ab" candidate_uuid: "decc879a4420405a955a509ba6b00cb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd"
I20260812 06:59:47.768247 22127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4b1fe76ac0b4e7a9080c8a6d1e092ab" candidate_uuid: "decc879a4420405a955a509ba6b00cb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f"
I20260812 06:59:47.768380 21901 raft_consensus.cc:3037] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:47.768491 22127 raft_consensus.cc:3037] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:47.770771 21901 raft_consensus.cc:2454] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate decc879a4420405a955a509ba6b00cb9 in term 1.
I20260812 06:59:47.770798 22127 raft_consensus.cc:2454] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate decc879a4420405a955a509ba6b00cb9 in term 1.
I20260812 06:59:47.771165 21570 leader_election.cc:302] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4f8bc5e74a243bca3e5c8759ea92fbd, decc879a4420405a955a509ba6b00cb9; no voters:
I20260812 06:59:47.771517 22251 raft_consensus.cc:2781] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:47.771754 22251 raft_consensus.cc:686] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 1 LEADER]: Becoming Leader. State: Replica: decc879a4420405a955a509ba6b00cb9, State: Running, Role: LEADER
I20260812 06:59:47.772290 22251 consensus_queue.cc:237] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 06:59:47.776409 21449 catalog_manager.cc:5495] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to decc879a4420405a955a509ba6b00cb9 (127.26.226.1). New cstate: current_term: 1 leader_uuid: "decc879a4420405a955a509ba6b00cb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } health_report { overall_health: HEALTHY } } }
I20260812 06:59:47.799000 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.793422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51488 (local address 127.26.226.1:40821)
0812 06:59:47.793613 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:47.793618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:47.793652 (+ 34us) server_negotiation.cc:407] Connection header received
0812 06:59:47.793850 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:47.793854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:47.793932 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:47.794063 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:47.794414 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.795146 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.796659 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.797190 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.797368 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.797903 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:47.798412 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:47.798590 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:47.798641 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 06:59:47.798727 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:59:47.807255 21901 raft_consensus.cc:1261] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd [term 1 FOLLOWER]: Refusing update from remote peer decc879a4420405a955a509ba6b00cb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:47.807264 22127 raft_consensus.cc:1261] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f [term 1 FOLLOWER]: Refusing update from remote peer decc879a4420405a955a509ba6b00cb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:47.808403 22251 consensus_queue.cc:1035] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:47.808748 22250 consensus_queue.cc:1035] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:59:47.810580 21980 tablet.cc:2369] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:47.820938 22260 mvcc.cc:205] Tried to move back new op lower bound from 7317577678048223232 to 7317577677919076352. Current Snapshot: MvccSnapshot[applied={T|T < 7317577678048223232}]
I20260812 06:59:47.836810 21448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:47.837164 21448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:47.841656 21448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Sending DeleteTablet for 3 replicas of tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab
I20260812 06:59:47.842832 21650 tablet_service.cc:1514] Processing DeleteTablet for tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:47 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:51484
I20260812 06:59:47.842975 22093 tablet_service.cc:1514] Processing DeleteTablet for tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:47 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:57398
I20260812 06:59:47.843575 21872 tablet_service.cc:1514] Processing DeleteTablet for tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:47 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:60458
I20260812 06:59:47.843809 22276 tablet_replica.cc:330] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: stopping tablet replica
I20260812 06:59:47.844473 22276 raft_consensus.cc:2229] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:59:47.845487 22276 raft_consensus.cc:2258] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:47.846735 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 21548
I20260812 06:59:47.848909 22278 tablet_replica.cc:330] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: stopping tablet replica
I20260812 06:59:47.849457 22278 raft_consensus.cc:2229] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:47.850088 22278 raft_consensus.cc:2258] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:47.852533 22276 ts_tablet_manager.cc:1905] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:47.853180 22278 ts_tablet_manager.cc:1905] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:47.860620 22278 ts_tablet_manager.cc:1918] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:47.860937 22278 log.cc:1196] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/b4b1fe76ac0b4e7a9080c8a6d1e092ab
I20260812 06:59:47.861101 22277 tablet_replica.cc:330] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: stopping tablet replica
I20260812 06:59:47.861582 22278 ts_tablet_manager.cc:1939] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P c4f8bc5e74a243bca3e5c8759ea92fbd: Deleting consensus metadata
I20260812 06:59:47.861985 22276 ts_tablet_manager.cc:1918] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:47.862247 22276 log.cc:1196] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/b4b1fe76ac0b4e7a9080c8a6d1e092ab
I20260812 06:59:47.862875 22276 ts_tablet_manager.cc:1939] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P decc879a4420405a955a509ba6b00cb9: Deleting consensus metadata
I20260812 06:59:47.863826 21422 catalog_manager.cc:4841] TS c4f8bc5e74a243bca3e5c8759ea92fbd (127.26.226.2:33433): tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab (table test-table [id=8f2d537d37714d058d12a44f4f18bc04]) successfully deleted
Aug 12 06:59:47 dist-test-slave-t3q3 krb5kdc[21380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 06:59:47.864708 22277 raft_consensus.cc:2229] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:47.864814 21426 catalog_manager.cc:4841] TS decc879a4420405a955a509ba6b00cb9 (127.26.226.1:40821): tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab (table test-table [id=8f2d537d37714d058d12a44f4f18bc04]) successfully deleted
I20260812 06:59:47.865367 22277 raft_consensus.cc:2258] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:47.867425 22277 ts_tablet_manager.cc:1905] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:47.873327 22277 ts_tablet_manager.cc:1918] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:47.873569 22277 log.cc:1196] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/b4b1fe76ac0b4e7a9080c8a6d1e092ab
I20260812 06:59:47.873737 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:47.848444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51502 (local address 127.26.226.1:40821)
0812 06:59:47.848669 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:47.848678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:47.851087 (+ 2409us) server_negotiation.cc:407] Connection header received
0812 06:59:47.851388 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:47.851395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:47.851484 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:47.851574 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:47.853837 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.855156 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.857267 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:47.857990 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:47.858098 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:47.865512 (+ 7414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:47.865525 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:47.865545 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:47.865602 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:47.869922 (+ 4320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:47.871099 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:47.871110 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:47.871115 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:47.871307 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:47.871803 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:47.871808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:47.871812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:47.872366 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:47.872680 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:47.873076 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 06:59:47.873348 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:59:47.874161 22277 ts_tablet_manager.cc:1939] T b4b1fe76ac0b4e7a9080c8a6d1e092ab P a9cf9c12d09c4a0eaaca8d23f31b061f: Deleting consensus metadata
I20260812 06:59:47.874725 21752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:47.883152 21422 catalog_manager.cc:4841] TS a9cf9c12d09c4a0eaaca8d23f31b061f (127.26.226.3:44297): tablet b4b1fe76ac0b4e7a9080c8a6d1e092ab (table test-table [id=8f2d537d37714d058d12a44f4f18bc04]) successfully deleted
I20260812 06:59:47.973392 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 21548
I20260812 06:59:48.003096 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 21769
Aug 12 06:59:48 dist-test-slave-t3q3 krb5kdc[21380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 06:59:48.018817 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.003956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60466 (local address 127.26.226.2:33433)
0812 06:59:48.004173 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.004179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.004306 (+ 127us) server_negotiation.cc:407] Connection header received
0812 06:59:48.004540 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.004545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.004612 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.004738 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.006127 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.006878 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.008199 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.008587 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.008740 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.013321 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.013336 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.013353 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.013388 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.016286 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.016957 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.016964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.016966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.017113 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.017468 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.017472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.017474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.017851 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.018109 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.018365 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.018531 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:59:48.019595 21963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:48.109429 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 21769
I20260812 06:59:48.136844 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 21988
Aug 12 06:59:48 dist-test-slave-t3q3 krb5kdc[21380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 06:59:48.153153 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.137730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.26.226.3:44297)
0812 06:59:48.137923 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.137931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.137953 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:59:48.138188 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.138194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.138271 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.138379 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.139866 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.140690 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.142080 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.142491 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.142641 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.147547 (+ 4906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.147563 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.147579 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.147620 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.150268 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.150958 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.150966 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.150969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.151111 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.151503 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.151508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.151513 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.151879 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.152206 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.152560 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.152805 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:59:48.153820 22202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:48.244760 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 21988
I20260812 06:59:48.272466 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 21401
I20260812 06:59:48.273716 21499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:59:48.385730 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 21401
I20260812 07:00:08.419597 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:43081
--webserver_interface=127.26.226.62
--webserver_port=37707
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:43081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:00:08.794014 22289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:08.794355 22289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:08.794521 22289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:08.802253 22289 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:00:08.802380 22289 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:00:08.802453 22289 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:00:08.802538 22289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:00:08.802624 22289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:08.802736 22289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:08.802824 22289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:00:08.802906 22289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:00:08.813637 22289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:43081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:43081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=37707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:08.815120 22289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:08.816606 22289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:08.844223 22300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:08.844178 22301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:08.844178 22303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:08.845571 22289 server_base.cc:1034] running on GCE node
I20260812 07:00:08.846555 22289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:08.848277 22289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:08.849578 22289 hybrid_clock.cc:648] HybridClock initialized: now 1786518008849500 us; error 63 us; skew 500 ppm
I20260812 07:00:08.850051 22289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:08 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518008, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:08.854197 22289 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:00:08.856156 22289 webserver.cc:460] Webserver started at http://127.26.226.62:37707/ using document root <none> and password file <none>
I20260812 07:00:08.857007 22289 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:08.857178 22289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:08.863533 22289 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 07:00:08.866616 22313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:08.867974 22289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:00:08.868233 22289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "32cd4d0f3b224ca0bd37eafc673fabce"
format_stamp: "Formatted at 2026-08-12 06:59:45 on dist-test-slave-t3q3"
server_key: "9fd59896015ead8084ef3493d0d11046"
server_key_iv: "1e168bc14164361488d6c847f98cf07d"
server_key_version: "encryptionkey@0"
I20260812 07:00:08.869310 22289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:08.887177 22289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:08.891717 22289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:08.892036 22289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:08.912779 22289 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:43081
I20260812 07:00:08.912822 22403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:43081 every 8 connection(s)
I20260812 07:00:08.914429 22289 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:00:08.917330 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 22289
I20260812 07:00:08.918393 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:40821
--local_ip_for_outbound_sockets=127.26.226.1
--tserver_master_addrs=127.26.226.62:43081
--webserver_port=33431
--webserver_interface=127.26.226.1
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:00:08.924244 22406 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:00:08.929625 22406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Bootstrap starting.
I20260812 07:00:08.951114 22406 log.cc:824] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:08.963503 22406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:00:08.964268 22406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Bootstrap complete.
I20260812 07:00:08.973834 22406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 07:00:08.974880 22406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32cd4d0f3b224ca0bd37eafc673fabce, State: Initialized, Role: FOLLOWER
I20260812 07:00:08.975526 22406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 07:00:08.975777 22406 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:00:08.975898 22406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:00:08.976135 22406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:00:08.979979 22406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 07:00:08.980602 22406 leader_election.cc:302] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32cd4d0f3b224ca0bd37eafc673fabce; no voters:
I20260812 07:00:08.981295 22406 leader_election.cc:288] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:00:08.981448 22411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:00:08.982746 22411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 2 LEADER]: Becoming Leader. State: Replica: 32cd4d0f3b224ca0bd37eafc673fabce, State: Running, Role: LEADER
I20260812 07:00:08.983397 22411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 07:00:08.983747 22406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:00:08.986784 22413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32cd4d0f3b224ca0bd37eafc673fabce. Latest consensus state: current_term: 2 leader_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } } }
I20260812 07:00:08.986816 22412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } } }
I20260812 07:00:08.987203 22413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: This master's current role is: LEADER
I20260812 07:00:08.987208 22412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: This master's current role is: LEADER
I20260812 07:00:08.996817 22419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:00:09.005134 22419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:00:09.006621 22419 catalog_manager.cc:1259] Loaded cluster ID: e4b0fa658d2b4833b00946a444fa394d
I20260812 07:00:09.006779 22419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:00:09.010919 22419 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:00:09.012583 22419 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Loaded TSK: 0
I20260812 07:00:09.043680 22419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Generated new TSK 1
I20260812 07:00:09.044131 22419 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:00:09.305467 22409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:09.305806 22409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:09.305992 22409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:09.313536 22409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:09.313691 22409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:09.313879 22409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 07:00:09.323660 22409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:40821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=33431
--tserver_master_addrs=127.26.226.62:43081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:09.325043 22409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:09.326365 22409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:09.341877 22439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:09.342000 22442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:09.342744 22438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:09.344115 22409 server_base.cc:1034] running on GCE node
I20260812 07:00:09.344885 22409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:09.345896 22409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:09.347162 22409 hybrid_clock.cc:648] HybridClock initialized: now 1786518009347120 us; error 55 us; skew 500 ppm
I20260812 07:00:09.347573 22409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:09 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518009, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:09.351578 22409 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 07:00:09.353291 22409 webserver.cc:460] Webserver started at http://127.26.226.1:33431/ using document root <none> and password file <none>
I20260812 07:00:09.354209 22409 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:09.354359 22409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:09.361383 22409 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 07:00:09.364527 22452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:09.366142 22409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:00:09.366312 22409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "decc879a4420405a955a509ba6b00cb9"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-t3q3"
server_key: "87756bbef3d8fc1c5e7f70267696c37f"
server_key_iv: "c0dd9dabca97ddf53639439d5c4d87d5"
server_key_version: "encryptionkey@0"
I20260812 07:00:09.367609 22409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:09.400848 22409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:09.405251 22409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:09.405560 22409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:09.406654 22409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:09.408049 22409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:09.408190 22409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:09.408325 22409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:09.408434 22409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:09.443926 22409 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:40821
I20260812 07:00:09.444022 22619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:40821 every 8 connection(s)
I20260812 07:00:09.445608 22409 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 07:00:09.454028 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 22409
I20260812 07:00:09.455590 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:33433
--local_ip_for_outbound_sockets=127.26.226.2
--tserver_master_addrs=127.26.226.62:43081
--webserver_port=37251
--webserver_interface=127.26.226.2
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:00:09 dist-test-slave-t3q3 krb5kdc[21380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518009, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:00:09.472711 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:09.448934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:53913)
0812 07:00:09.449868 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:09.449914 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:09.451676 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:09.453135 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:09.453142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:09.453959 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:00:09.454750 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:09.454780 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:09.456676 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:09.456685 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:09.458895 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:09.458912 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:09.460002 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:09.460012 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:09.460350 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:09.461147 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:00:09.461186 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:00:09.463968 (+ 2782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:00:09.467777 (+ 3809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:09.467788 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:09.467806 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:09.468356 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:09.468863 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:09.468869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:09.468873 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:09.469081 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:09.469908 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:00:09.469917 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:00:09.470284 (+ 367us) client_negotiation.cc:769] Sending connection context
0812 07:00:09.470586 (+ 302us) client_negotiation.cc:240] Negotiation successful
0812 07:00:09.471559 (+ 973us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":389,"threads_started":1}
I20260812 07:00:09.472711 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:09.448728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:53913 (local address 127.26.226.62:43081)
0812 07:00:09.449330 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:09.449343 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:09.450056 (+ 713us) server_negotiation.cc:407] Connection header received
0812 07:00:09.452263 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:09.452275 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:09.452611 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:09.452900 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:09.454951 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:09.456485 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:09.459079 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:09.459749 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:09.459889 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:09.464314 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:09.464323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:09.464339 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:09.464384 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:09.467586 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:09.468528 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:09.468536 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:09.468542 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:09.468723 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:09.469223 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:09.469228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:09.469231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:09.469742 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:09.469978 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:09.470831 (+ 853us) server_negotiation.cc:299] Negotiation successful
0812 07:00:09.471254 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":256,"threads_started":1}
I20260812 07:00:09.475981 22622 heartbeater.cc:347] Connected to a master server at 127.26.226.62:43081
I20260812 07:00:09.476366 22622 heartbeater.cc:464] Registering TS with master...
I20260812 07:00:09.477285 22622 heartbeater.cc:511] Master 127.26.226.62:43081 requested a full tablet report, sending...
I20260812 07:00:09.479713 22345 ts_manager.cc:194] Registered new tserver with Master: decc879a4420405a955a509ba6b00cb9 (127.26.226.1:40821)
I20260812 07:00:09.481289 22345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:53913
W20260812 07:00:09.816038 22629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:09.816442 22629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:09.816656 22629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:09.823899 22629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:09.824014 22629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:09.824267 22629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 07:00:09.834177 22629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:33433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=37251
--tserver_master_addrs=127.26.226.62:43081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:09.835605 22629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:09.837016 22629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:09.851521 22639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:09.851660 22638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:09.851569 22642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:09.852866 22629 server_base.cc:1034] running on GCE node
I20260812 07:00:09.853494 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:09.854491 22629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:09.855710 22629 hybrid_clock.cc:648] HybridClock initialized: now 1786518009855638 us; error 65 us; skew 500 ppm
I20260812 07:00:09.856043 22629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:09 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518009, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:09.859488 22629 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 07:00:09.861034 22629 webserver.cc:460] Webserver started at http://127.26.226.2:37251/ using document root <none> and password file <none>
I20260812 07:00:09.861701 22629 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:09.861819 22629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:09.867190 22629 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:00:09.869936 22653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:09.871374 22629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:00:09.871534 22629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-t3q3"
server_key: "ff3d591753afba4e845a16ab4326cb9e"
server_key_iv: "f2d42abd338a6fc7b327f8c4957c43f6"
server_key_version: "encryptionkey@0"
I20260812 07:00:09.872876 22629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:09.902290 22629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:09.906875 22629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:09.907191 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:09.908316 22629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:09.909716 22629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:09.909807 22629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:09.909981 22629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:09.910081 22629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:09.950501 22629 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33433
I20260812 07:00:09.950565 22828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33433 every 8 connection(s)
I20260812 07:00:09.951930 22629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 07:00:09.956534 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 22629
I20260812 07:00:09.957172 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:44297
--local_ip_for_outbound_sockets=127.26.226.3
--tserver_master_addrs=127.26.226.62:43081
--webserver_port=41305
--webserver_interface=127.26.226.3
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:00:09 dist-test-slave-t3q3 krb5kdc[21380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518009, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:00:09.972936 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:09.954876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:50097 (local address 127.26.226.62:43081)
0812 07:00:09.955061 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:09.955065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:09.956210 (+ 1145us) server_negotiation.cc:407] Connection header received
0812 07:00:09.958242 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:09.958248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:09.958354 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:09.958473 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:09.959934 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:09.960730 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:09.962315 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:09.962745 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:09.962892 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:09.966634 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:09.966640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:09.966643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:09.966673 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:09.969233 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:09.970190 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:09.970195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:09.970197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:09.970335 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:09.970923 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:09.970928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:09.970930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:09.971268 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:09.971395 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:09.972340 (+ 945us) server_negotiation.cc:299] Negotiation successful
0812 07:00:09.972524 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:00:09.974563 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:09.955291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:50097)
0812 07:00:09.956008 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:09.956044 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:09.957964 (+ 1920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:09.958708 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:09.958716 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:09.959217 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:00:09.959750 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:09.959766 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:09.960911 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:09.960915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:09.962136 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:09.962148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:09.962977 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:09.962985 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:09.963222 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:09.963947 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:00:09.963970 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:00:09.966395 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:00:09.969383 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:09.969394 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:09.969413 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:09.970049 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:09.970469 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:09.970477 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:09.970482 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:09.970755 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:09.971427 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:00:09.971437 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:00:09.971877 (+ 440us) client_negotiation.cc:769] Sending connection context
0812 07:00:09.972213 (+ 336us) client_negotiation.cc:240] Negotiation successful
0812 07:00:09.972840 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":286,"threads_started":1}
I20260812 07:00:09.976133 22829 heartbeater.cc:347] Connected to a master server at 127.26.226.62:43081
I20260812 07:00:09.976518 22829 heartbeater.cc:464] Registering TS with master...
I20260812 07:00:09.977317 22829 heartbeater.cc:511] Master 127.26.226.62:43081 requested a full tablet report, sending...
I20260812 07:00:09.978953 22345 ts_manager.cc:194] Registered new tserver with Master: c4f8bc5e74a243bca3e5c8759ea92fbd (127.26.226.2:33433)
I20260812 07:00:09.979776 22345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:50097
W20260812 07:00:10.315429 22836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:10.315764 22836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:10.315936 22836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:10.323189 22836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:10.323342 22836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:10.323524 22836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 07:00:10.334205 22836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:44297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=41305
--tserver_master_addrs=127.26.226.62:43081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:10.335561 22836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:10.336987 22836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:10.350204 22841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:10.350270 22842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:10.350327 22844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:10.351570 22836 server_base.cc:1034] running on GCE node
I20260812 07:00:10.352345 22836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:10.353286 22836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:10.354578 22836 hybrid_clock.cc:648] HybridClock initialized: now 1786518010354519 us; error 34 us; skew 500 ppm
I20260812 07:00:10.354920 22836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:10 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518010, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:10.358135 22836 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 07:00:10.359647 22836 webserver.cc:460] Webserver started at http://127.26.226.3:41305/ using document root <none> and password file <none>
I20260812 07:00:10.360419 22836 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:10.360598 22836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:10.365923 22836 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:00:10.368624 22855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:10.369942 22836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:00:10.370131 22836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f"
format_stamp: "Formatted at 2026-08-12 06:59:47 on dist-test-slave-t3q3"
server_key: "cb72053c3b09ff1515081f93ca1d86e5"
server_key_iv: "73dc76ee489048fa30e8bc69c5c02d94"
server_key_version: "encryptionkey@0"
I20260812 07:00:10.371147 22836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:10.399398 22836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:10.403937 22836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:10.404278 22836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:10.405355 22836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:10.406754 22836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:10.406888 22836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:10.407032 22836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:10.407135 22836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:10.442090 22836 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:44297
I20260812 07:00:10.442185 23024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:44297 every 8 connection(s)
I20260812 07:00:10.443696 22836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 07:00:10.452291 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 22836
Aug 12 07:00:10 dist-test-slave-t3q3 krb5kdc[21380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518010, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:00:10.466730 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.446708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:40763 (local address 127.26.226.62:43081)
0812 07:00:10.446944 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.446948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.448352 (+ 1404us) server_negotiation.cc:407] Connection header received
0812 07:00:10.450237 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.450243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.450330 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.450458 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:10.451961 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.452845 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.454904 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.455360 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.455508 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.459351 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:10.459359 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:10.459364 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:10.459402 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:10.463007 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:10.463955 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:10.463959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:10.463962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:10.464166 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:10.464851 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:10.464855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:10.464857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:10.465139 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:10.465259 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.466141 (+ 882us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.466299 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:00:10.467756 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.447252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:40763)
0812 07:00:10.448179 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:10.448224 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:10.449948 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:10.450704 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:10.450712 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:10.451228 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:00:10.451774 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.451791 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.453048 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.453054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.454736 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.454749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.455610 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.455617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.455849 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.456608 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:00:10.456630 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:00:10.459116 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:00:10.463152 (+ 4036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:10.463161 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:10.463176 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:10.463828 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:10.464292 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:10.464297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:10.464302 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:10.464548 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:10.465318 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:00:10.465327 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:00:10.465740 (+ 413us) client_negotiation.cc:769] Sending connection context
0812 07:00:10.466086 (+ 346us) client_negotiation.cc:240] Negotiation successful
0812 07:00:10.466587 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":367,"threads_started":1}
I20260812 07:00:10.469147 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.453368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.26.226.62:43081)
0812 07:00:10.454158 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.454165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.454184 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:00:10.454350 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.454355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.454421 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.454538 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:10.456161 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.456913 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.458345 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.458985 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.459118 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.462881 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:10.462887 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:10.462890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:10.462928 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:10.466258 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:10.467068 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:10.467074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:10.467079 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:10.467192 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:10.467599 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:10.467605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:10.467608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:10.467859 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:10.468169 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.468534 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.468758 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":525,"threads_started":1}
I20260812 07:00:10.470533 23025 heartbeater.cc:347] Connected to a master server at 127.26.226.62:43081
I20260812 07:00:10.470867 23025 heartbeater.cc:464] Registering TS with master...
I20260812 07:00:10.471612 23025 heartbeater.cc:511] Master 127.26.226.62:43081 requested a full tablet report, sending...
I20260812 07:00:10.473322 22344 ts_manager.cc:194] Registered new tserver with Master: a9cf9c12d09c4a0eaaca8d23f31b061f (127.26.226.3:44297)
I20260812 07:00:10.474474 22344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:40763
I20260812 07:00:10.484742 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:00:10.485651 22622 heartbeater.cc:503] Master 127.26.226.62:43081 was elected leader, sending a full tablet report...
I20260812 07:00:10.491868 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.485682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.26.226.62:43081)
0812 07:00:10.485894 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.485900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.486475 (+ 575us) server_negotiation.cc:407] Connection header received
0812 07:00:10.486735 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.486739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.486868 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.487028 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:00:10.487481 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.488265 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.489790 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.490255 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.490399 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.490906 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:00:10.491175 (+ 269us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 07:00:10.491599 (+ 424us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43326: token expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 07:00:10.492017 23032 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43326: token expired
W20260812 07:00:10.492499 22217 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.26.226.62:43081)
I20260812 07:00:10.506306 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.493053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43334 (local address 127.26.226.62:43081)
0812 07:00:10.493225 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.493231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.493342 (+ 111us) server_negotiation.cc:407] Connection header received
0812 07:00:10.493519 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.493528 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.493601 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.493708 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:10.495123 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.495817 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.497392 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.497804 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.497949 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.500722 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:10.500733 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:10.500736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:10.500767 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:10.503501 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:10.504281 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:10.504288 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:10.504291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:10.504425 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:10.504832 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:10.504842 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:10.504844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:10.505149 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:10.505421 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.505758 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.505976 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:00:10.509898 22344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:00:10.517320 22344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:00:10.554445 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.529460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33433 (local address 127.0.0.1:55384)
0812 07:00:10.530264 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:10.535902 (+ 5638us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:10.536222 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:10.543446 (+ 7224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:10.543454 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:10.543491 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:10.543978 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.544000 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.546671 (+ 2671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.546680 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.550075 (+ 3395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.550101 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.552882 (+ 2781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.552892 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.553265 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.553356 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 07:00:10.553523 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 07:00:10.553815 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":379,"threads_started":1}
I20260812 07:00:10.555567 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.529695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55384 (local address 127.26.226.2:33433)
0812 07:00:10.542562 (+ 12867us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.542577 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.542621 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:00:10.542771 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.542785 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.543151 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.543428 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:10.544290 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.546435 (+ 2145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.550394 (+ 3959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.552526 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.554737 (+ 2211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.554960 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.555057 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.555263 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12595,"thread_start_us":13966,"threads_started":1}
I20260812 07:00:10.559868 22722 tablet_service.cc:1467] Processing CreateTablet for tablet e8f74ac8b8014966bb2cae8e1dfea5ef (DEFAULT_TABLE table=test-table [id=bd6245eb57d8420fb31450e69dedb257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:00:10.561293 22722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8f74ac8b8014966bb2cae8e1dfea5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:10.564432 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.530685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:44297 (local address 127.0.0.1:51186)
0812 07:00:10.555576 (+ 24891us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:10.555598 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:10.556054 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:10.556825 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:10.556830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:10.556857 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:10.557231 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.557271 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.559100 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.559105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.561333 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.561349 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.563522 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.563529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.563712 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.563748 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:00:10.563863 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 07:00:10.564032 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23915,"thread_start_us":24021,"threads_started":1}
I20260812 07:00:10.564507 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.531340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51186 (local address 127.26.226.3:44297)
0812 07:00:10.552946 (+ 21606us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.552960 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.555739 (+ 2779us) server_negotiation.cc:407] Connection header received
0812 07:00:10.556308 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.556317 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.556586 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.556787 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:10.557512 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.558864 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.561524 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.563003 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.563137 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.563370 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.563943 (+ 573us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.564139 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21341,"thread_start_us":21673,"threads_started":1}
I20260812 07:00:10.564864 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.530179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:40821 (local address 127.0.0.1:35326)
0812 07:00:10.553856 (+ 23677us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:10.553884 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:10.554070 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:10.557333 (+ 3263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:10.557338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:10.557369 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:10.557712 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.557723 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.560083 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.560090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.562175 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.562192 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.564099 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.564106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.564293 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.564324 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:00:10.564391 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 07:00:10.564548 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23483,"thread_start_us":24233,"threads_started":1}
I20260812 07:00:10.565721 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.530637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35326 (local address 127.26.226.1:40821)
0812 07:00:10.556738 (+ 26101us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.556749 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.556779 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:00:10.556879 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.556890 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.557124 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.557316 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:10.557909 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.559892 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.562375 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.563827 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.565002 (+ 1175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.565234 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.565315 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.565475 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25925,"thread_start_us":25771,"threads_started":1}
I20260812 07:00:10.568882 22931 tablet_service.cc:1467] Processing CreateTablet for tablet e8f74ac8b8014966bb2cae8e1dfea5ef (DEFAULT_TABLE table=test-table [id=bd6245eb57d8420fb31450e69dedb257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:00:10.569228 22528 tablet_service.cc:1467] Processing CreateTablet for tablet e8f74ac8b8014966bb2cae8e1dfea5ef (DEFAULT_TABLE table=test-table [id=bd6245eb57d8420fb31450e69dedb257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:00:10.570523 22931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8f74ac8b8014966bb2cae8e1dfea5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:10.570550 22528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8f74ac8b8014966bb2cae8e1dfea5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:10.580840 23053 tablet_bootstrap.cc:492] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: Bootstrap starting.
I20260812 07:00:10.584873 23053 tablet_bootstrap.cc:654] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:10.585997 23053 log.cc:824] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:10.587000 23056 tablet_bootstrap.cc:492] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: Bootstrap starting.
I20260812 07:00:10.588096 23057 tablet_bootstrap.cc:492] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: Bootstrap starting.
I20260812 07:00:10.590868 23053 tablet_bootstrap.cc:492] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: No bootstrap required, opened a new log
I20260812 07:00:10.591164 23053 ts_tablet_manager.cc:1397] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260812 07:00:10.591745 23057 tablet_bootstrap.cc:654] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:10.591958 23056 tablet_bootstrap.cc:654] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:10.593010 23056 log.cc:824] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:10.593083 23057 log.cc:824] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:10.596349 23056 tablet_bootstrap.cc:492] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: No bootstrap required, opened a new log
I20260812 07:00:10.596359 23057 tablet_bootstrap.cc:492] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: No bootstrap required, opened a new log
I20260812 07:00:10.596649 23057 ts_tablet_manager.cc:1397] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260812 07:00:10.596649 23056 ts_tablet_manager.cc:1397] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 07:00:10.600140 23053 raft_consensus.cc:348] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 07:00:10.600566 23053 raft_consensus.cc:374] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:10.600716 23053 raft_consensus.cc:729] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f8bc5e74a243bca3e5c8759ea92fbd, State: Initialized, Role: FOLLOWER
I20260812 07:00:10.601384 23053 consensus_queue.cc:260] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 07:00:10.602630 22829 heartbeater.cc:503] Master 127.26.226.62:43081 was elected leader, sending a full tablet report...
I20260812 07:00:10.603330 23053 ts_tablet_manager.cc:1428] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260812 07:00:10.607426 23057 raft_consensus.cc:348] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 07:00:10.607905 23057 raft_consensus.cc:374] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:10.608004 23057 raft_consensus.cc:729] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9cf9c12d09c4a0eaaca8d23f31b061f, State: Initialized, Role: FOLLOWER
I20260812 07:00:10.608731 23057 consensus_queue.cc:260] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 07:00:10.608866 23056 raft_consensus.cc:348] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 07:00:10.609386 23056 raft_consensus.cc:374] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:10.609560 23056 raft_consensus.cc:729] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: decc879a4420405a955a509ba6b00cb9, State: Initialized, Role: FOLLOWER
I20260812 07:00:10.610188 23025 heartbeater.cc:503] Master 127.26.226.62:43081 was elected leader, sending a full tablet report...
I20260812 07:00:10.610318 23056 consensus_queue.cc:260] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 07:00:10.611001 23057 ts_tablet_manager.cc:1428] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260812 07:00:10.611994 23056 ts_tablet_manager.cc:1428] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260812 07:00:10.660511 23069 raft_consensus.cc:482] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:00:10.660877 23069 raft_consensus.cc:504] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 07:00:10.662798 23069 leader_election.cc:288] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4f8bc5e74a243bca3e5c8759ea92fbd (127.26.226.2:33433), a9cf9c12d09c4a0eaaca8d23f31b061f (127.26.226.3:44297)
I20260812 07:00:10.669605 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.663164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:54753 (local address 127.26.226.2:33433)
0812 07:00:10.663369 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.663374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.664044 (+ 670us) server_negotiation.cc:407] Connection header received
0812 07:00:10.664286 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.664290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.664350 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.664470 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:10.665220 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.665951 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.667746 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.668621 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.668816 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.668895 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.669192 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.669324 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:00:10.669732 23070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.663140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33433 (local address 127.26.226.1:54753)
0812 07:00:10.663914 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:10.663937 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:10.664123 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:10.664499 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:10.664503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:10.664532 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:10.665049 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.665061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.666107 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.666112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.667555 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.667571 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.668845 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.668850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.669014 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.669039 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:00:10.669163 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 07:00:10.669293 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":1,"thread_start_us":434,"threads_started":1}
I20260812 07:00:10.670373 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.664051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:44297 (local address 127.26.226.1:53151)
0812 07:00:10.665090 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:10.665105 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:10.665259 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:10.665587 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:10.665592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:10.665616 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:10.665944 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.665952 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.666950 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.666957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.668449 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:10.668462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.669637 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.669643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:10.669853 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.669882 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:00:10.669975 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 07:00:10.670099 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":300,"threads_started":1}
I20260812 07:00:10.670480 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.664365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:53151 (local address 127.26.226.3:44297)
0812 07:00:10.664596 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.664604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.665199 (+ 595us) server_negotiation.cc:407] Connection header received
0812 07:00:10.665397 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.665402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.665467 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.665587 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:10.666106 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.666813 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.668617 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.669395 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.669605 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.669681 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.670057 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.670199 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:00:10.670903 22758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f74ac8b8014966bb2cae8e1dfea5ef" candidate_uuid: "decc879a4420405a955a509ba6b00cb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" is_pre_election: true
I20260812 07:00:10.671423 22758 raft_consensus.cc:2454] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate decc879a4420405a955a509ba6b00cb9 in term 0.
I20260812 07:00:10.671540 22959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f74ac8b8014966bb2cae8e1dfea5ef" candidate_uuid: "decc879a4420405a955a509ba6b00cb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" is_pre_election: true
I20260812 07:00:10.672178 22453 leader_election.cc:302] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4f8bc5e74a243bca3e5c8759ea92fbd, decc879a4420405a955a509ba6b00cb9; no voters:
I20260812 07:00:10.672178 22959 raft_consensus.cc:2454] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate decc879a4420405a955a509ba6b00cb9 in term 0.
I20260812 07:00:10.672590 23069 raft_consensus.cc:2781] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:00:10.672783 23069 raft_consensus.cc:482] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:00:10.672897 23069 raft_consensus.cc:3037] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:10.675294 23069 raft_consensus.cc:504] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 07:00:10.675961 23069 leader_election.cc:288] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [CANDIDATE]: Term 1 election: Requested vote from peers c4f8bc5e74a243bca3e5c8759ea92fbd (127.26.226.2:33433), a9cf9c12d09c4a0eaaca8d23f31b061f (127.26.226.3:44297)
I20260812 07:00:10.676501 22758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f74ac8b8014966bb2cae8e1dfea5ef" candidate_uuid: "decc879a4420405a955a509ba6b00cb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd"
I20260812 07:00:10.676539 22959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f74ac8b8014966bb2cae8e1dfea5ef" candidate_uuid: "decc879a4420405a955a509ba6b00cb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f"
I20260812 07:00:10.676713 22758 raft_consensus.cc:3037] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:10.676779 22959 raft_consensus.cc:3037] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:10.679173 22959 raft_consensus.cc:2454] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate decc879a4420405a955a509ba6b00cb9 in term 1.
I20260812 07:00:10.679177 22758 raft_consensus.cc:2454] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate decc879a4420405a955a509ba6b00cb9 in term 1.
I20260812 07:00:10.679636 22453 leader_election.cc:302] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9cf9c12d09c4a0eaaca8d23f31b061f, decc879a4420405a955a509ba6b00cb9; no voters:
I20260812 07:00:10.679975 23069 raft_consensus.cc:2781] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:00:10.680476 23069 raft_consensus.cc:686] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 1 LEADER]: Becoming Leader. State: Replica: decc879a4420405a955a509ba6b00cb9, State: Running, Role: LEADER
I20260812 07:00:10.681133 23069 consensus_queue.cc:237] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } }
I20260812 07:00:10.686494 22344 catalog_manager.cc:5495] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to decc879a4420405a955a509ba6b00cb9 (127.26.226.1). New cstate: current_term: 1 leader_uuid: "decc879a4420405a955a509ba6b00cb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "decc879a4420405a955a509ba6b00cb9" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 40821 } health_report { overall_health: HEALTHY } } }
W20260812 07:00:10.695717 23027 tablet.cc:2369] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:00:10.698129 22623 tablet.cc:2369] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:00:10.704228 22830 tablet.cc:2369] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:00:10.725443 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.719585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35336 (local address 127.26.226.1:40821)
0812 07:00:10.719813 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.719820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.719906 (+ 86us) server_negotiation.cc:407] Connection header received
0812 07:00:10.720197 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.720202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.720296 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.720462 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:00:10.720879 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.721637 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.723113 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.723520 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.723674 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.724259 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:00:10.724799 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:00:10.724990 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.725046 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.725146 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 07:00:10.738258 22959 raft_consensus.cc:1261] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f [term 1 FOLLOWER]: Refusing update from remote peer decc879a4420405a955a509ba6b00cb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:00:10.738260 22758 raft_consensus.cc:1261] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd [term 1 FOLLOWER]: Refusing update from remote peer decc879a4420405a955a509ba6b00cb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:00:10.739465 23072 consensus_queue.cc:1035] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 44297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:00:10.739908 23069 consensus_queue.cc:1035] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:00:10.755952 23078 mvcc.cc:205] Tried to move back new op lower bound from 7317577771970650112 to 7317577771754860544. Current Snapshot: MvccSnapshot[applied={T|T < 7317577771970650112}]
I20260812 07:00:10.769266 22344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:00:10.769588 22344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:00:10.773795 22344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Sending DeleteTablet for 3 replicas of tablet e8f74ac8b8014966bb2cae8e1dfea5ef
I20260812 07:00:10.775185 22722 tablet_service.cc:1514] Processing DeleteTablet for tablet e8f74ac8b8014966bb2cae8e1dfea5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:00:10 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:55384
I20260812 07:00:10.775455 22528 tablet_service.cc:1514] Processing DeleteTablet for tablet e8f74ac8b8014966bb2cae8e1dfea5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:00:10 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:35326
I20260812 07:00:10.775496 22931 tablet_service.cc:1514] Processing DeleteTablet for tablet e8f74ac8b8014966bb2cae8e1dfea5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:00:10 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:51186
I20260812 07:00:10.781147 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 22409
I20260812 07:00:10.787963 23090 tablet_replica.cc:330] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: stopping tablet replica
I20260812 07:00:10.788662 23090 raft_consensus.cc:2229] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:00:10.789623 23090 raft_consensus.cc:2258] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:00:10.792119 23090 ts_tablet_manager.cc:1905] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:00:10.797298 23093 tablet_replica.cc:330] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: stopping tablet replica
I20260812 07:00:10.797750 23093 raft_consensus.cc:2229] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:00:10.798360 23093 raft_consensus.cc:2258] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:00:10.801409 23093 ts_tablet_manager.cc:1905] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:00:10.806550 23090 ts_tablet_manager.cc:1918] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:00:10.807327 23090 log.cc:1196] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/e8f74ac8b8014966bb2cae8e1dfea5ef
I20260812 07:00:10.809445 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.783125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35350 (local address 127.26.226.1:40821)
0812 07:00:10.783351 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.783359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.783383 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:00:10.783458 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.783465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.783669 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.783788 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:10.786583 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.788082 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.790429 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.791163 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.791307 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.796732 (+ 5425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:10.796748 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:10.796766 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:10.796816 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:10.803463 (+ 6647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:10.806819 (+ 3356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:10.806830 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:10.806833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:10.807072 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:10.807572 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:10.807578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:10.807582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:10.808121 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:10.808430 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.808777 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.809055 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 07:00:10.810384 23090 ts_tablet_manager.cc:1939] T e8f74ac8b8014966bb2cae8e1dfea5ef P decc879a4420405a955a509ba6b00cb9: Deleting consensus metadata
I20260812 07:00:10.810556 22613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:10.812012 23089 tablet_replica.cc:330] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: stopping tablet replica
I20260812 07:00:10.812331 23093 ts_tablet_manager.cc:1918] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:00:10.812541 23093 log.cc:1196] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/e8f74ac8b8014966bb2cae8e1dfea5ef
I20260812 07:00:10.812620 23089 raft_consensus.cc:2229] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:00:10.813200 23093 ts_tablet_manager.cc:1939] T e8f74ac8b8014966bb2cae8e1dfea5ef P a9cf9c12d09c4a0eaaca8d23f31b061f: Deleting consensus metadata
I20260812 07:00:10.813251 23089 raft_consensus.cc:2258] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:00:10.815223 22315 catalog_manager.cc:4841] TS a9cf9c12d09c4a0eaaca8d23f31b061f (127.26.226.3:44297): tablet e8f74ac8b8014966bb2cae8e1dfea5ef (table test-table [id=bd6245eb57d8420fb31450e69dedb257]) successfully deleted
I20260812 07:00:10.815516 23089 ts_tablet_manager.cc:1905] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:00:10.822618 23089 ts_tablet_manager.cc:1918] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:00:10.822825 23089 log.cc:1196] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/e8f74ac8b8014966bb2cae8e1dfea5ef
I20260812 07:00:10.823397 23089 ts_tablet_manager.cc:1939] T e8f74ac8b8014966bb2cae8e1dfea5ef P c4f8bc5e74a243bca3e5c8759ea92fbd: Deleting consensus metadata
I20260812 07:00:10.825608 22315 catalog_manager.cc:4841] TS c4f8bc5e74a243bca3e5c8759ea92fbd (127.26.226.2:33433): tablet e8f74ac8b8014966bb2cae8e1dfea5ef (table test-table [id=bd6245eb57d8420fb31450e69dedb257]) successfully deleted
I20260812 07:00:10.921882 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 22409
I20260812 07:00:10.929106 22319 catalog_manager.cc:4841] TS decc879a4420405a955a509ba6b00cb9 (127.26.226.1:40821): tablet e8f74ac8b8014966bb2cae8e1dfea5ef (table test-table [id=bd6245eb57d8420fb31450e69dedb257]) successfully deleted
I20260812 07:00:10.951625 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 22629
I20260812 07:00:10.967708 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:10.952535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.26.226.2:33433)
0812 07:00:10.952732 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:10.952738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:10.952759 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:00:10.952915 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:10.952921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:10.953041 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:10.953153 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:10.954590 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.955761 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.957268 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:10.957923 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:10.958043 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:10.960900 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:10.960913 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:10.960932 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:10.960972 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:10.964779 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:10.965520 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:10.965529 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:10.965533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:10.965679 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:10.966092 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:10.966097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:10.966100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:10.966588 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:10.966905 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:10.967145 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 07:00:10.967359 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 07:00:10.968458 22823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:11.060153 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 22629
I20260812 07:00:11.086072 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 22836
I20260812 07:00:11.100564 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:11.086913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.26.226.3:44297)
0812 07:00:11.087110 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:11.087115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:11.087136 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:00:11.087301 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:11.087305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:11.087373 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:11.087461 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:11.089013 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:11.089779 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:11.091003 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:11.091394 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:11.091490 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:11.094176 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:11.094191 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:11.094207 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:11.094246 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:11.097422 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:11.098207 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:11.098215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:11.098217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:11.098362 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:11.098793 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:11.098796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:11.098798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:11.099211 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:11.099440 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:11.099716 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 07:00:11.099888 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:00:11.101279 23020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:11.191159 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 22836
I20260812 07:00:11.216936 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 22289
I20260812 07:00:11.218169 22395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:11.336377 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 22289
I20260812 07:00:31.368791 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:43081
--webserver_interface=127.26.226.62
--webserver_port=37707
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:43081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:00:31.733217 23101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:31.733579 23101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:31.733758 23101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:31.741707 23101 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:00:31.741850 23101 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:00:31.741941 23101 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:00:31.742038 23101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:00:31.742168 23101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:31.742296 23101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:31.742369 23101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:00:31.742470 23101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:00:31.753330 23101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:43081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:43081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=37707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:31.755082 23101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:31.756691 23101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:31.770272 23110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:31.770367 23113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:31.770244 23108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:31.770711 23101 server_base.cc:1034] running on GCE node
I20260812 07:00:31.771896 23101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:31.773488 23101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:31.774744 23101 hybrid_clock.cc:648] HybridClock initialized: now 1786518031774656 us; error 67 us; skew 500 ppm
I20260812 07:00:31.775100 23101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:31 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518031, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:31.778573 23101 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:00:31.780179 23101 webserver.cc:460] Webserver started at http://127.26.226.62:37707/ using document root <none> and password file <none>
I20260812 07:00:31.780905 23101 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:31.781044 23101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:31.786798 23101 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:00:31.789947 23124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:31.791333 23101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260812 07:00:31.791546 23101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "32cd4d0f3b224ca0bd37eafc673fabce"
format_stamp: "Formatted at 2026-08-12 06:59:45 on dist-test-slave-t3q3"
server_key: "9fd59896015ead8084ef3493d0d11046"
server_key_iv: "1e168bc14164361488d6c847f98cf07d"
server_key_version: "encryptionkey@0"
I20260812 07:00:31.792626 23101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:31.810752 23101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:31.815224 23101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:31.815541 23101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:31.836609 23101 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:43081
I20260812 07:00:31.836676 23198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:43081 every 8 connection(s)
I20260812 07:00:31.838402 23101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:00:31.845996 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 23101
I20260812 07:00:31.847105 23200 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:00:31.847287 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:40821
--local_ip_for_outbound_sockets=127.26.226.1
--tserver_master_addrs=127.26.226.62:43081
--webserver_port=33431
--webserver_interface=127.26.226.1
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:00:31.853410 23200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Bootstrap starting.
I20260812 07:00:31.879218 23200 log.cc:824] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:31.896813 23200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:00:31.897517 23200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Bootstrap complete.
I20260812 07:00:31.907393 23200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 07:00:31.908420 23200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32cd4d0f3b224ca0bd37eafc673fabce, State: Initialized, Role: FOLLOWER
I20260812 07:00:31.909132 23200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 07:00:31.909394 23200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:00:31.909524 23200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:00:31.909730 23200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 2 FOLLOWER]: Advancing to term 3
I20260812 07:00:31.913895 23200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 07:00:31.914428 23200 leader_election.cc:302] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32cd4d0f3b224ca0bd37eafc673fabce; no voters:
I20260812 07:00:31.915246 23200 leader_election.cc:288] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 07:00:31.915380 23204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 3 FOLLOWER]: Leader election won for term 3
I20260812 07:00:31.916676 23204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [term 3 LEADER]: Becoming Leader. State: Replica: 32cd4d0f3b224ca0bd37eafc673fabce, State: Running, Role: LEADER
I20260812 07:00:31.917327 23204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } }
I20260812 07:00:31.917622 23200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:00:31.920047 23208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32cd4d0f3b224ca0bd37eafc673fabce. Latest consensus state: current_term: 3 leader_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } } }
I20260812 07:00:31.920121 23205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32cd4d0f3b224ca0bd37eafc673fabce" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43081 } } }
I20260812 07:00:31.920480 23208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: This master's current role is: LEADER
I20260812 07:00:31.920557 23205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce [sys.catalog]: This master's current role is: LEADER
I20260812 07:00:31.921705 23216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:00:31.937341 23216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:00:31.938939 23216 catalog_manager.cc:1259] Loaded cluster ID: e4b0fa658d2b4833b00946a444fa394d
I20260812 07:00:31.939123 23216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:00:31.943387 23216 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:00:31.945149 23216 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Loaded TSK: 0
I20260812 07:00:31.945482 23216 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 32cd4d0f3b224ca0bd37eafc673fabce: Loaded TSK: 1
F20260812 07:00:31.946681 23216 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc21effc9ad google::LogMessage::Fail() at ??:0
@ 0x7fc21efff88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc21effc4de google::LogMessage::Flush() at ??:0
@ 0x7fc21effda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc222d607d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc222d4f73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc234a2b1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc234a2a783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc234ab62a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fc234ab60c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc234b005e9 std::function<>::operator()() at ??:0
@ 0x7fc234a2ecdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc234a2cc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc234ab4610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc234ab4439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc221c48345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc221c4b91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc221c4b799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc221c26eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc224f006db start_thread at ??:0
@ 0x7fc21de9661f clone at ??:0
W20260812 07:00:32.278157 23202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:32.278438 23202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:32.278609 23202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:32.291314 23202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:32.291435 23202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:32.291636 23202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 07:00:32.301920 23202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:40821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=33431
--tserver_master_addrs=127.26.226.62:43081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:32.303328 23202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:32.304760 23202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:32.318324 23265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:32.318356 23264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:32.318327 23267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:32.319275 23202 server_base.cc:1034] running on GCE node
I20260812 07:00:32.319916 23202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:32.320904 23202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:32.322135 23202 hybrid_clock.cc:648] HybridClock initialized: now 1786518032322082 us; error 62 us; skew 500 ppm
I20260812 07:00:32.322544 23202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:32 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:32.330071 23202 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 07:00:32.331537 23202 webserver.cc:460] Webserver started at http://127.26.226.1:33431/ using document root <none> and password file <none>
I20260812 07:00:32.332321 23202 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:32.332476 23202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:32.337993 23202 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:00:32.340831 23280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:32.342191 23202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:00:32.342381 23202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "decc879a4420405a955a509ba6b00cb9"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-t3q3"
server_key: "87756bbef3d8fc1c5e7f70267696c37f"
server_key_iv: "c0dd9dabca97ddf53639439d5c4d87d5"
server_key_version: "encryptionkey@0"
I20260812 07:00:32.343833 23202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:32.357645 23202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:32.361850 23202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:32.362141 23202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:32.363206 23202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:32.364648 23202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:32.364780 23202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:32.364920 23202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:32.365022 23202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:32.399936 23202 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:40821
I20260812 07:00:32.400039 23448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:40821 every 8 connection(s)
I20260812 07:00:32.401489 23202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 07:00:32.404397 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 23202
I20260812 07:00:32.405052 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:33433
--local_ip_for_outbound_sockets=127.26.226.2
--tserver_master_addrs=127.26.226.62:43081
--webserver_port=37251
--webserver_interface=127.26.226.2
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:00:32.406730 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:32.404673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:37039)
0812 07:00:32.405561 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:32.405878 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":377,"threads_started":1}
W20260812 07:00:32.407418 23449 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:43081 (0 consecutive failures): Network error: Failed to ping master at 127.26.226.62:43081: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
I20260812 07:00:32.408401 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:32.407992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:59063)
0812 07:00:32.408224 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:32.408296 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260812 07:00:32.409286 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:32.408938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:44189)
0812 07:00:32.409084 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:32.409170 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 07:00:32.409520 23449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:00:32.756390 23456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:32.756711 23456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:32.756891 23456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:32.764160 23456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:32.764278 23456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:32.764393 23456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 07:00:32.774055 23456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:33433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=37251
--tserver_master_addrs=127.26.226.62:43081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:32.775254 23456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:32.776648 23456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:32.789871 23471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:32.789940 23466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:32.790177 23465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:32.790773 23456 server_base.cc:1034] running on GCE node
I20260812 07:00:32.791450 23456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:32.792416 23456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:32.793617 23456 hybrid_clock.cc:648] HybridClock initialized: now 1786518032793578 us; error 38 us; skew 500 ppm
I20260812 07:00:32.793932 23456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:32 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:32.797140 23456 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 07:00:32.798615 23456 webserver.cc:460] Webserver started at http://127.26.226.2:37251/ using document root <none> and password file <none>
I20260812 07:00:32.799283 23456 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:32.799433 23456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:32.804517 23456 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:00:32.807214 23482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:32.808480 23456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:00:32.808656 23456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "c4f8bc5e74a243bca3e5c8759ea92fbd"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-t3q3"
server_key: "ff3d591753afba4e845a16ab4326cb9e"
server_key_iv: "f2d42abd338a6fc7b327f8c4957c43f6"
server_key_version: "encryptionkey@0"
I20260812 07:00:32.809624 23456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:32.825898 23456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:32.830322 23456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:32.830619 23456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:32.831640 23456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:32.833068 23456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:32.833202 23456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:32.833344 23456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:32.833443 23456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:32.868578 23456 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33433
I20260812 07:00:32.868637 23646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33433 every 8 connection(s)
I20260812 07:00:32.870155 23456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 07:00:32.874946 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:32.873211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:51505)
0812 07:00:32.873924 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:32.874155 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":289,"threads_started":1}
W20260812 07:00:32.875658 23649 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:43081 (0 consecutive failures): Network error: Failed to ping master at 127.26.226.62:43081: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
I20260812 07:00:32.876524 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:32.876221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:32967)
0812 07:00:32.876351 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:32.876418 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:00:32.877331 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:32.877034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:53047)
0812 07:00:32.877151 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:32.877227 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:00:32.877588 23649 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:00:32.878110 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 23456
I20260812 07:00:32.878733 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:44297
--local_ip_for_outbound_sockets=127.26.226.3
--tserver_master_addrs=127.26.226.62:43081
--webserver_port=41305
--webserver_interface=127.26.226.3
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:00:33.228194 23653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:33.228571 23653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:33.228767 23653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:33.236243 23653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:33.236423 23653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:33.236624 23653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 07:00:33.246798 23653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:44297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=41305
--tserver_master_addrs=127.26.226.62:43081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:33.248303 23653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:33.249683 23653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:33.263439 23660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:33.263504 23659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:33.263535 23662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:33.264895 23653 server_base.cc:1034] running on GCE node
I20260812 07:00:33.265524 23653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:33.266429 23653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:33.267642 23653 hybrid_clock.cc:648] HybridClock initialized: now 1786518033267597 us; error 28 us; skew 500 ppm
I20260812 07:00:33.268010 23653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:33 dist-test-slave-t3q3 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518033, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:33.271313 23653 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 07:00:33.272856 23653 webserver.cc:460] Webserver started at http://127.26.226.3:41305/ using document root <none> and password file <none>
I20260812 07:00:33.273512 23653 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:33.273618 23653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:33.278848 23653 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:00:33.281455 23671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:33.282728 23653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:00:33.282917 23653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "a9cf9c12d09c4a0eaaca8d23f31b061f"
format_stamp: "Formatted at 2026-08-12 06:59:47 on dist-test-slave-t3q3"
server_key: "cb72053c3b09ff1515081f93ca1d86e5"
server_key_iv: "73dc76ee489048fa30e8bc69c5c02d94"
server_key_version: "encryptionkey@0"
I20260812 07:00:33.283897 23653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:33.314821 23653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:33.319067 23653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:33.319373 23653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:33.320490 23653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:33.321871 23653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:33.322018 23653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:33.322150 23653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:33.322242 23653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:33.356977 23653 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:44297
I20260812 07:00:33.357069 23842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:44297 every 8 connection(s)
I20260812 07:00:33.358588 23653 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 07:00:33.361701 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 23653
I20260812 07:00:33.364477 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:33.362255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:32889)
0812 07:00:33.363147 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:33.363473 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":348,"threads_started":1}
W20260812 07:00:33.365207 23845 heartbeater.cc:644] Failed to heartbeat to 127.26.226.62:43081 (0 consecutive failures): Network error: Failed to ping master at 127.26.226.62:43081: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
I20260812 07:00:33.366030 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:33.365695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:37563)
0812 07:00:33.365859 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:33.365925 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 07:00:33.366856 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:33.366527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:42309)
0812 07:00:33.366669 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:33.366751 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 07:00:33.367100 23845 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:00:33.410862 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:33.410029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:33789)
0812 07:00:33.410508 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:33.410611 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":236,"threads_started":1}
I20260812 07:00:33.878950 23851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:33.878135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:40423)
0812 07:00:33.878606 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:33.878716 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":233,"threads_started":1}
I20260812 07:00:34.368603 23852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:34.367720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:33507)
0812 07:00:34.368271 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:34.368365 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":242,"threads_started":1}
I20260812 07:00:34.412734 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:34.411864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:50685)
0812 07:00:34.412403 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:34.412512 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":298,"threads_started":1}
I20260812 07:00:34.880606 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:34.879855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:38643)
0812 07:00:34.880367 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:34.880446 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":294,"threads_started":1}
I20260812 07:00:35.370537 23858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:35.369637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:37447)
0812 07:00:35.370232 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:35.370348 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":296,"threads_started":1}
I20260812 07:00:35.414652 23859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:35.413749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:55597)
0812 07:00:35.414332 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:35.414475 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":331,"threads_started":1}
I20260812 07:00:35.882164 23860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:35.881450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:41059)
0812 07:00:35.881909 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:35.881990 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":238,"threads_started":1}
I20260812 07:00:36.372422 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:36.371561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:42497)
0812 07:00:36.372149 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:36.372242 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":288,"threads_started":1}
I20260812 07:00:36.416365 23862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:36.415566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:49995)
0812 07:00:36.416020 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:36.416163 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":232,"threads_started":1}
I20260812 07:00:36.883898 23863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:36.883106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:58815)
0812 07:00:36.883621 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:36.883701 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":280,"threads_started":1}
I20260812 07:00:37.374166 23864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:37.373434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:40725)
0812 07:00:37.373902 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:37.373988 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":239,"threads_started":1}
I20260812 07:00:37.418182 23865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:37.417407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:45461)
0812 07:00:37.417920 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:37.418007 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":276,"threads_started":1}
I20260812 07:00:37.885712 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:37.884926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:55047)
0812 07:00:37.885429 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:37.885524 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":252,"threads_started":1}
I20260812 07:00:38.375991 23867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:38.375226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:56039)
0812 07:00:38.375732 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:38.375817 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":282,"threads_started":1}
I20260812 07:00:38.419965 23868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:38.419171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:43911)
0812 07:00:38.419689 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:38.419791 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":254,"threads_started":1}
I20260812 07:00:38.887902 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:38.886912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:51363)
0812 07:00:38.887541 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:38.887709 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":353,"threads_started":1}
I20260812 07:00:39.377740 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:39.376987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:41235)
0812 07:00:39.377470 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:39.377567 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":245,"threads_started":1}
I20260812 07:00:39.421875 23873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:39.421044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:35331)
0812 07:00:39.421573 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:39.421656 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":245,"threads_started":1}
I20260812 07:00:39.890097 23876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:39.888989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:45221)
0812 07:00:39.889753 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:39.889841 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":435,"threads_started":1}
I20260812 07:00:40.379539 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:40.378722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:54579)
0812 07:00:40.379230 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:40.379324 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":225,"threads_started":1}
I20260812 07:00:40.423696 23879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:40.422801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:49075)
0812 07:00:40.423245 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:40.423495 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":230,"threads_started":1}
I20260812 07:00:40.891713 23881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:40.891014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:58853)
0812 07:00:40.891455 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:40.891535 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":234,"threads_started":1}
I20260812 07:00:41.381639 23882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:41.380704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:37909)
0812 07:00:41.381373 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:41.381462 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":403,"threads_started":1}
I20260812 07:00:41.425531 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:41.424657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:43335)
0812 07:00:41.425197 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:41.425302 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":251,"threads_started":1}
I20260812 07:00:41.893683 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:41.892648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:39113)
0812 07:00:41.893371 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:41.893491 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":322,"threads_started":1}
I20260812 07:00:42.383366 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:42.382531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:38213)
0812 07:00:42.383071 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:42.383182 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":249,"threads_started":1}
I20260812 07:00:42.427338 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:42.426515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:34135)
0812 07:00:42.427077 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:42.427158 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":344,"threads_started":1}
I20260812 07:00:42.895411 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:42.894667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:37785)
0812 07:00:42.895145 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:42.895232 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":269,"threads_started":1}
I20260812 07:00:43.385143 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:43.384396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:59815)
0812 07:00:43.384879 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:43.384967 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":253,"threads_started":1}
I20260812 07:00:43.429116 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:43.428397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:41227)
0812 07:00:43.428840 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:43.428931 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":218,"threads_started":1}
I20260812 07:00:43.897411 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:43.896529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:44815)
0812 07:00:43.897086 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:43.897219 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":314,"threads_started":1}
I20260812 07:00:44.386968 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:44.386209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:58523)
0812 07:00:44.386661 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:44.386773 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":255,"threads_started":1}
I20260812 07:00:44.430841 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:44.430115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:37261)
0812 07:00:44.430575 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:44.430663 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":235,"threads_started":1}
I20260812 07:00:44.899241 23900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:44.898469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:39319)
0812 07:00:44.898976 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:44.899068 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":254,"threads_started":1}
I20260812 07:00:45.388777 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:45.387907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:58113)
0812 07:00:45.388526 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:45.388616 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":337,"threads_started":1}
I20260812 07:00:45.432570 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:45.431728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:45387)
0812 07:00:45.432281 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:45.432377 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":241,"threads_started":1}
I20260812 07:00:45.901146 23903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:45.900319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:55309)
0812 07:00:45.900844 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:45.900950 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":258,"threads_started":1}
I20260812 07:00:46.390481 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:46.389665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:41685)
0812 07:00:46.390172 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:46.390277 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":251,"threads_started":1}
I20260812 07:00:46.434266 23908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:46.433529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:58315)
0812 07:00:46.433968 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:46.434063 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":223,"threads_started":1}
I20260812 07:00:46.902956 23910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:46.902206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:35585)
0812 07:00:46.902680 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:46.902770 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":249,"threads_started":1}
I20260812 07:00:47.392249 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:47.391425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:37687)
0812 07:00:47.391881 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:47.391980 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":226,"threads_started":1}
I20260812 07:00:47.436299 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:47.435403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.1:45465)
0812 07:00:47.435939 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:47.436034 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":304,"threads_started":1}
I20260812 07:00:47.904983 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:47.904254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.2:60481)
0812 07:00:47.904701 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:47.904801 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":220,"threads_started":1}
I20260812 07:00:48.363247 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 23202
I20260812 07:00:48.382588 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:48.364523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37452 (local address 127.26.226.1:40821)
0812 07:00:48.365573 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:48.365598 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:48.365639 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:00:48.367234 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:48.367243 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:48.367678 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:48.368042 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:48.369717 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:48.370966 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:48.372444 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:48.373145 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:48.373364 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:48.376207 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:48.376216 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:48.376232 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:48.376269 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:48.379077 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:48.379838 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:48.379847 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:48.379856 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:48.380035 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:48.380520 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:48.380525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:48.380528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:48.381064 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:48.381398 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:48.381889 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 07:00:48.382305 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":462,"threads_started":1}
I20260812 07:00:48.384163 23441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:48.394134 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:48.393322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43081 (local address 127.26.226.3:32775)
0812 07:00:48.393857 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:48.393953 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.62:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":283,"threads_started":1}
I20260812 07:00:48.469856 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 23202
I20260812 07:00:48.496537 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 23456
I20260812 07:00:48.515322 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:48.497518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37368 (local address 127.26.226.2:33433)
0812 07:00:48.498028 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:48.498043 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:48.498080 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:00:48.499566 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:48.499576 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:48.500014 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:48.500441 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:48.502066 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:48.503313 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:48.505009 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:48.505716 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:48.505917 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:48.508748 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:48.508756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:48.508770 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:48.508812 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:48.511824 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:48.512660 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:48.512666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:48.512673 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:48.512853 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:48.513294 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:48.513298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:48.513300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:48.513808 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:48.514178 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:48.514576 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 07:00:48.515006 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":237,"threads_started":1}
I20260812 07:00:48.516781 23638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:48.605046 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 23456
I20260812 07:00:48.633375 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 23653
I20260812 07:00:48.652820 23922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:48.634478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45136 (local address 127.26.226.3:44297)
0812 07:00:48.635128 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:48.635143 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:48.635189 (+ 46us) server_negotiation.cc:407] Connection header received
0812 07:00:48.636991 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:48.637003 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:48.637467 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:48.637794 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:48.639300 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:48.640685 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:48.642178 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:48.642909 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:48.643068 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:48.645930 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:48.645939 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:48.645956 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:48.646003 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:48.649371 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:48.650185 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:48.650192 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:48.650199 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:48.650377 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:48.650756 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:48.650759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:48.650761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:48.651312 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:48.651547 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:48.651991 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 07:00:48.652491 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":320,"threads_started":1}
I20260812 07:00:48.654386 23830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:48.744670 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 23653
2026-08-12T07:00:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65046 ms)
[----------] 30 tests from SecurityITest (314409 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:00:48 dist-test-slave-t3q3 krb5kdc[23930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:00:48 dist-test-slave-t3q3 krb5kdc[23930](info): set up 2 sockets
Aug 12 07:00:48 dist-test-slave-t3q3 krb5kdc[23930](info): commencing operation
krb5kdc: starting...
W20260812 07:00:50.135352 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.341s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:00:50 dist-test-slave-t3q3 krb5kdc[23930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:00:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:00:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:00:50.473475 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:46787
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:46787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:00:50.835449 23948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:50.835760 23948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:50.835944 23948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:50.843281 23948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:00:50.843400 23948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:50.843503 23948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:50.843621 23948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:00:50.843694 23948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:00:50.853745 23948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:46787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:46787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:50.855160 23948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:50.856508 23948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:50.868829 23966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:50.868774 23959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:50.868868 23960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:50.869450 23948 server_base.cc:1034] running on GCE node
I20260812 07:00:50.870328 23948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:50.871752 23948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:50.873099 23948 hybrid_clock.cc:648] HybridClock initialized: now 1786518050873056 us; error 51 us; skew 500 ppm
I20260812 07:00:50.873497 23948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:50 dist-test-slave-t3q3 krb5kdc[23930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518050, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:50.876820 23948 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:00:50.878495 23948 webserver.cc:460] Webserver started at http://127.26.226.62:34055/ using document root <none> and password file <none>
I20260812 07:00:50.879247 23948 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:50.879406 23948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:50.879740 23948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:00:50.882983 23948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "198dc9d4535245aebc209a783547edd2"
format_stamp: "Formatted at 2026-08-12 07:00:50 on dist-test-slave-t3q3"
server_key: "7386f0e905747f03e4f76b7672ff62b6"
server_key_iv: "d751c7ef669e9dd86b4b68aaf04547b7"
server_key_version: "encryptionkey@0"
I20260812 07:00:50.883781 23948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "198dc9d4535245aebc209a783547edd2"
format_stamp: "Formatted at 2026-08-12 07:00:50 on dist-test-slave-t3q3"
server_key: "7386f0e905747f03e4f76b7672ff62b6"
server_key_iv: "d751c7ef669e9dd86b4b68aaf04547b7"
server_key_version: "encryptionkey@0"
I20260812 07:00:50.889258 23948 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 07:00:50.892807 23977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:50.894307 23948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 07:00:50.894533 23948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "198dc9d4535245aebc209a783547edd2"
format_stamp: "Formatted at 2026-08-12 07:00:50 on dist-test-slave-t3q3"
server_key: "7386f0e905747f03e4f76b7672ff62b6"
server_key_iv: "d751c7ef669e9dd86b4b68aaf04547b7"
server_key_version: "encryptionkey@0"
I20260812 07:00:50.894788 23948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:50.917529 23948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:50.921761 23948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:50.922068 23948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:50.941681 23948 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:46787
I20260812 07:00:50.941726 24062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:46787 every 8 connection(s)
I20260812 07:00:50.943130 23948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:00:50.947007 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 23948
I20260812 07:00:50.947217 24064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:50.947261 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:00:50.947974 27528 external_mini_cluster.cc:1428] Setting key 59acdac32f5e5529cedd415c58d5489c
I20260812 07:00:50.961771 24064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2: Bootstrap starting.
Aug 12 07:00:50 dist-test-slave-t3q3 krb5kdc[23930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:00:50.965824 24064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:50.966951 24064 log.cc:824] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:50.970080 24064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2: No bootstrap required, opened a new log
I20260812 07:00:50.973687 24069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:50.950139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55082 (local address 127.26.226.62:46787)
0812 07:00:50.951001 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:50.951013 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:50.951054 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:00:50.953435 (+ 2381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:50.953460 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:50.953875 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:50.954232 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:50.956169 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:50.957495 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:50.959517 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:50.960280 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:50.960414 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:50.965748 (+ 5334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:50.965757 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:50.965773 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:50.965817 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:50.969267 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:50.970036 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:50.970052 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:50.970060 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:50.970281 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:50.970702 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:50.970707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:50.970710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:50.971246 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:50.971579 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:50.972128 (+ 549us) server_negotiation.cc:299] Negotiation successful
0812 07:00:50.972609 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":420,"threads_started":1}
I20260812 07:00:50.978380 24064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198dc9d4535245aebc209a783547edd2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46787 } }
I20260812 07:00:50.978778 24064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:50.978889 24064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 198dc9d4535245aebc209a783547edd2, State: Initialized, Role: FOLLOWER
I20260812 07:00:50.979457 24064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198dc9d4535245aebc209a783547edd2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46787 } }
I20260812 07:00:50.979717 24064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:00:50.979868 24064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:00:50.980108 24064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:50.982878 24064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198dc9d4535245aebc209a783547edd2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46787 } }
I20260812 07:00:50.983464 24064 leader_election.cc:302] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 198dc9d4535245aebc209a783547edd2; no voters:
I20260812 07:00:50.984278 24064 leader_election.cc:288] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:00:50.984397 24071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:00:50.985944 24071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [term 1 LEADER]: Becoming Leader. State: Replica: 198dc9d4535245aebc209a783547edd2, State: Running, Role: LEADER
I20260812 07:00:50.986518 24064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:00:50.986541 24071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198dc9d4535245aebc209a783547edd2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46787 } }
I20260812 07:00:50.989564 24079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 198dc9d4535245aebc209a783547edd2. Latest consensus state: current_term: 1 leader_uuid: "198dc9d4535245aebc209a783547edd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198dc9d4535245aebc209a783547edd2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46787 } } }
I20260812 07:00:50.989677 24073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "198dc9d4535245aebc209a783547edd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198dc9d4535245aebc209a783547edd2" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 46787 } } }
I20260812 07:00:50.990146 24073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:00:50.990141 24079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:00:50.992465 24094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:00:50.998540 24094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:00:51.011209 24094 catalog_manager.cc:1347] Generated new cluster ID: 0a35725922f0453cbabf548d23e94055
I20260812 07:00:51.011348 24094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:00:51.021775 24094 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:00:51.023701 24094 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:00:51.031471 24094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2: Generated new TSK 0
I20260812 07:00:51.032585 24094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 07:00:51.125540 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46787
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:00:51.481431 24119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:51.481721 24119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:51.481915 24119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:51.489254 24119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:51.489369 24119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:51.489491 24119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 07:00:51.499646 24119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:51.500991 24119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:51.502342 24119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:51.515575 24125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:51.515686 24124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:51.515681 24128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:51.516605 24119 server_base.cc:1034] running on GCE node
I20260812 07:00:51.517216 24119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:51.518394 24119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:51.519603 24119 hybrid_clock.cc:648] HybridClock initialized: now 1786518051519551 us; error 41 us; skew 500 ppm
I20260812 07:00:51.519884 24119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:51 dist-test-slave-t3q3 krb5kdc[23930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518051, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:51.523237 24119 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 07:00:51.524809 24119 webserver.cc:460] Webserver started at http://127.26.226.1:44329/ using document root <none> and password file <none>
I20260812 07:00:51.525532 24119 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:51.525687 24119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:51.526021 24119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:00:51.529134 24119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "a4294e0d0c094c1eb2a8bc9cfe457728"
format_stamp: "Formatted at 2026-08-12 07:00:51 on dist-test-slave-t3q3"
server_key: "e6808f6fff207726be19f3cfb507baac"
server_key_iv: "3c1d502dabdc5dea752e8eea65fa22fc"
server_key_version: "encryptionkey@0"
I20260812 07:00:51.529953 24119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "a4294e0d0c094c1eb2a8bc9cfe457728"
format_stamp: "Formatted at 2026-08-12 07:00:51 on dist-test-slave-t3q3"
server_key: "e6808f6fff207726be19f3cfb507baac"
server_key_iv: "3c1d502dabdc5dea752e8eea65fa22fc"
server_key_version: "encryptionkey@0"
I20260812 07:00:51.535507 24119 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260812 07:00:51.539083 24137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:51.540458 24119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:00:51.540642 24119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "a4294e0d0c094c1eb2a8bc9cfe457728"
format_stamp: "Formatted at 2026-08-12 07:00:51 on dist-test-slave-t3q3"
server_key: "e6808f6fff207726be19f3cfb507baac"
server_key_iv: "3c1d502dabdc5dea752e8eea65fa22fc"
server_key_version: "encryptionkey@0"
I20260812 07:00:51.540859 24119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:51.558424 24119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:51.562397 24119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:51.562713 24119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:51.563782 24119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:51.565266 24119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:51.565375 24119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:51.565533 24119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:51.565608 24119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:51.601755 24119 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:46461
I20260812 07:00:51.601773 24307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:46461 every 8 connection(s)
I20260812 07:00:51.603112 24119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 07:00:51.604596 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 24119
I20260812 07:00:51.604823 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 07:00:51.605659 27528 external_mini_cluster.cc:1428] Setting key ccaaa545d50a5d0c9433d9e59f2d9086
Aug 12 07:00:51 dist-test-slave-t3q3 krb5kdc[23930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518051, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:00:51.624294 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:51.605916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:35055 (local address 127.26.226.62:46787)
0812 07:00:51.606692 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:51.606698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:51.607679 (+ 981us) server_negotiation.cc:407] Connection header received
0812 07:00:51.609691 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:51.609696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:51.609781 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:51.609919 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:51.611479 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:51.612311 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:51.613890 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:51.614308 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:51.614448 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:51.618143 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:51.618149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:51.618154 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:51.618200 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:51.620976 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:51.621836 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:51.621840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:51.621842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:51.621972 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:51.622518 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:51.622522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:51.622524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:51.622786 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:51.622904 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:51.623718 (+ 814us) server_negotiation.cc:299] Negotiation successful
0812 07:00:51.623897 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":513,"threads_started":1}
I20260812 07:00:51.625293 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:51.606629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:46787 (local address 127.26.226.1:35055)
0812 07:00:51.607514 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:51.607560 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:51.609406 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:51.610181 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:51.610189 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:51.610719 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:00:51.611290 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:51.611308 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:51.612478 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:51.612485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:51.613738 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:51.613751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:51.614567 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:51.614574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:51.614810 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:51.615549 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:00:51.615578 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:00:51.617912 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:00:51.621129 (+ 3217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:51.621139 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:51.621172 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:51.621692 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:51.622104 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:51.622110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:51.622115 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:51.622400 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:51.622926 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:00:51.622938 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:00:51.623356 (+ 418us) client_negotiation.cc:769] Sending connection context
0812 07:00:51.623602 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 07:00:51.624001 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":348,"threads_started":1}
I20260812 07:00:51.626936 24308 heartbeater.cc:347] Connected to a master server at 127.26.226.62:46787
I20260812 07:00:51.627300 24308 heartbeater.cc:464] Registering TS with master...
I20260812 07:00:51.628305 24308 heartbeater.cc:511] Master 127.26.226.62:46787 requested a full tablet report, sending...
I20260812 07:00:51.630580 24014 ts_manager.cc:194] Registered new tserver with Master: a4294e0d0c094c1eb2a8bc9cfe457728 (127.26.226.1:46461)
I20260812 07:00:51.632105 24014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:35055
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 07:00:51.683384 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46787
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:00:52.051836 24320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:52.052237 24320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:52.052346 24320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:52.060177 24320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:52.060298 24320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:52.060433 24320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 07:00:52.071751 24320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:52.073231 24320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:52.074586 24320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:52.089620 24330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:52.089810 24326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:52.089767 24328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:52.090368 24320 server_base.cc:1034] running on GCE node
I20260812 07:00:52.091043 24320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:52.091995 24320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:52.093255 24320 hybrid_clock.cc:648] HybridClock initialized: now 1786518052093182 us; error 57 us; skew 500 ppm
I20260812 07:00:52.093600 24320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:52 dist-test-slave-t3q3 krb5kdc[23930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518052, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:52.101455 24320 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 07:00:52.103169 24320 webserver.cc:460] Webserver started at http://127.26.226.2:39941/ using document root <none> and password file <none>
I20260812 07:00:52.103928 24320 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:52.104135 24320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:52.104498 24320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:00:52.107511 24320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "64b6d552e703413f8170ef8936cf625b"
format_stamp: "Formatted at 2026-08-12 07:00:52 on dist-test-slave-t3q3"
server_key: "a5e3682754b28abc22d3b19644f4ed67"
server_key_iv: "948c81e5f4f3a22877993a818154b488"
server_key_version: "encryptionkey@0"
I20260812 07:00:52.108456 24320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "64b6d552e703413f8170ef8936cf625b"
format_stamp: "Formatted at 2026-08-12 07:00:52 on dist-test-slave-t3q3"
server_key: "a5e3682754b28abc22d3b19644f4ed67"
server_key_iv: "948c81e5f4f3a22877993a818154b488"
server_key_version: "encryptionkey@0"
I20260812 07:00:52.113826 24320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:00:52.117411 24341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:52.118719 24320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:00:52.118911 24320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "64b6d552e703413f8170ef8936cf625b"
format_stamp: "Formatted at 2026-08-12 07:00:52 on dist-test-slave-t3q3"
server_key: "a5e3682754b28abc22d3b19644f4ed67"
server_key_iv: "948c81e5f4f3a22877993a818154b488"
server_key_version: "encryptionkey@0"
I20260812 07:00:52.119127 24320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:52.131351 24320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:52.135661 24320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:52.135983 24320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:52.137007 24320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:52.138388 24320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:52.138506 24320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:52.138664 24320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:52.138741 24320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:52.173179 24320 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33519
I20260812 07:00:52.173228 24513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33519 every 8 connection(s)
I20260812 07:00:52.174480 24320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 07:00:52.180598 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 24320
I20260812 07:00:52.180811 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 07:00:52.181284 27528 external_mini_cluster.cc:1428] Setting key 8fc9420d7e98a09608f99bbc6edec74d
Aug 12 07:00:52 dist-test-slave-t3q3 krb5kdc[23930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518052, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:00:52.198258 24520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.177297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:38631 (local address 127.26.226.62:46787)
0812 07:00:52.177755 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:52.177761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:52.178819 (+ 1058us) server_negotiation.cc:407] Connection header received
0812 07:00:52.180642 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:52.180649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:52.180746 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:52.180880 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:52.182677 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.183617 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.185270 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.186077 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.186231 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.190111 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:52.190120 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:52.190129 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:52.190176 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:52.194037 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:52.195038 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:52.195048 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:52.195054 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:52.195232 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:52.195727 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:52.195735 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:52.195741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:52.196199 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:52.196368 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:52.197246 (+ 878us) server_negotiation.cc:299] Negotiation successful
0812 07:00:52.197676 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":336,"thread_start_us":207,"threads_started":1}
I20260812 07:00:52.199119 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.177938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:46787 (local address 127.26.226.2:38631)
0812 07:00:52.178641 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:52.178695 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:52.180372 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:52.181114 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:52.181121 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:52.181701 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:00:52.182504 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:52.182529 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.183788 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.183792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:52.185102 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:52.185120 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.186345 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.186352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:52.186552 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.187346 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:00:52.187397 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:00:52.189849 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:00:52.194218 (+ 4369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:52.194231 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:52.194251 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:52.194887 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:52.195352 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:52.195359 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:52.195365 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:52.195598 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:52.196342 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:00:52.196352 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:00:52.196820 (+ 468us) client_negotiation.cc:769] Sending connection context
0812 07:00:52.197109 (+ 289us) client_negotiation.cc:240] Negotiation successful
0812 07:00:52.197723 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":281,"threads_started":1}
I20260812 07:00:52.200701 24517 heartbeater.cc:347] Connected to a master server at 127.26.226.62:46787
I20260812 07:00:52.201043 24517 heartbeater.cc:464] Registering TS with master...
I20260812 07:00:52.201798 24517 heartbeater.cc:511] Master 127.26.226.62:46787 requested a full tablet report, sending...
I20260812 07:00:52.203369 24014 ts_manager.cc:194] Registered new tserver with Master: 64b6d552e703413f8170ef8936cf625b (127.26.226.2:33519)
I20260812 07:00:52.204339 24014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:38631
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 07:00:52.254827 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46787
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:00:52.611181 24528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:52.611493 24528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:52.611663 24528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:52.619009 24528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:52.619176 24528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:52.619350 24528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 07:00:52.629303 24528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:46787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:52.630582 24528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:52.631932 24528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:00:52.635957 24308 heartbeater.cc:503] Master 127.26.226.62:46787 was elected leader, sending a full tablet report...
W20260812 07:00:52.645902 24534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:52.646054 24535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:52.646132 24544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:52.647121 24528 server_base.cc:1034] running on GCE node
I20260812 07:00:52.647781 24528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:52.648774 24528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:52.649982 24528 hybrid_clock.cc:648] HybridClock initialized: now 1786518052649945 us; error 28 us; skew 500 ppm
I20260812 07:00:52.650307 24528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:52 dist-test-slave-t3q3 krb5kdc[23930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518052, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:52.653614 24528 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 07:00:52.655023 24528 webserver.cc:460] Webserver started at http://127.26.226.3:38735/ using document root <none> and password file <none>
I20260812 07:00:52.655707 24528 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:52.655860 24528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:52.656286 24528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:00:52.659162 24528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "616c96aff9a949e2a3df9b114ccf1c8a"
format_stamp: "Formatted at 2026-08-12 07:00:52 on dist-test-slave-t3q3"
server_key: "13780ebbc49e913a8dda66d4db9d04a1"
server_key_iv: "a9a6f6bd9a7ca27f06d32780989db00e"
server_key_version: "encryptionkey@0"
I20260812 07:00:52.659950 24528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "616c96aff9a949e2a3df9b114ccf1c8a"
format_stamp: "Formatted at 2026-08-12 07:00:52 on dist-test-slave-t3q3"
server_key: "13780ebbc49e913a8dda66d4db9d04a1"
server_key_iv: "a9a6f6bd9a7ca27f06d32780989db00e"
server_key_version: "encryptionkey@0"
I20260812 07:00:52.665096 24528 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 07:00:52.668633 24553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:52.669934 24528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:00:52.670112 24528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "616c96aff9a949e2a3df9b114ccf1c8a"
format_stamp: "Formatted at 2026-08-12 07:00:52 on dist-test-slave-t3q3"
server_key: "13780ebbc49e913a8dda66d4db9d04a1"
server_key_iv: "a9a6f6bd9a7ca27f06d32780989db00e"
server_key_version: "encryptionkey@0"
I20260812 07:00:52.670351 24528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:52.691196 24528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:52.695865 24528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:52.696264 24528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:52.697394 24528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:52.699074 24528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:52.699193 24528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:52.699347 24528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:52.699440 24528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:52.751345 24528 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:45979
I20260812 07:00:52.751442 24754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:45979 every 8 connection(s)
I20260812 07:00:52.752796 24528 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 07:00:52.758257 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 24528
I20260812 07:00:52.758481 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 07:00:52.758953 27528 external_mini_cluster.cc:1428] Setting key 39522491eeb4bb10a7f04cfef1b72e8b
Aug 12 07:00:52 dist-test-slave-t3q3 krb5kdc[23930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518052, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:00:52.773746 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.755484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:54467 (local address 127.26.226.62:46787)
0812 07:00:52.756012 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:52.756017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:52.756975 (+ 958us) server_negotiation.cc:407] Connection header received
0812 07:00:52.759019 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:52.759026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:52.759124 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:52.759255 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:52.761215 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.762149 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.763751 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.764228 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.764368 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.767812 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:52.767817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:52.767820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:52.767854 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:52.770420 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:52.771172 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:52.771179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:52.771182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:52.771315 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:52.771762 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:52.771767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:52.771770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:52.772038 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:52.772258 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:52.773067 (+ 809us) server_negotiation.cc:299] Negotiation successful
0812 07:00:52.773298 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":298,"threads_started":1}
I20260812 07:00:52.775035 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.755996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:46787 (local address 127.26.226.3:54467)
0812 07:00:52.756797 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:52.756839 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:52.758716 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:52.759567 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:52.759580 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:52.760265 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:00:52.761022 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:52.761055 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.762330 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.762334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:52.763578 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:52.763591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.764472 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.764480 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:52.764718 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.765367 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:00:52.765394 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:00:52.767585 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:00:52.770572 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:52.770581 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:52.770598 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:52.771042 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:52.771436 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:52.771441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:52.771445 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:52.771634 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:52.772230 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:00:52.772238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:00:52.772582 (+ 344us) client_negotiation.cc:769] Sending connection context
0812 07:00:52.772901 (+ 319us) client_negotiation.cc:240] Negotiation successful
0812 07:00:52.773475 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":279,"threads_started":1}
I20260812 07:00:52.776446 24755 heartbeater.cc:347] Connected to a master server at 127.26.226.62:46787
I20260812 07:00:52.776809 24755 heartbeater.cc:464] Registering TS with master...
I20260812 07:00:52.777549 24755 heartbeater.cc:511] Master 127.26.226.62:46787 requested a full tablet report, sending...
I20260812 07:00:52.779062 24014 ts_manager.cc:194] Registered new tserver with Master: 616c96aff9a949e2a3df9b114ccf1c8a (127.26.226.3:45979)
I20260812 07:00:52.779884 24014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:54467
I20260812 07:00:52.787266 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:00:52.805583 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.792146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.26.226.62:46787)
0812 07:00:52.792329 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:52.792334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:52.792839 (+ 505us) server_negotiation.cc:407] Connection header received
0812 07:00:52.792992 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:52.792995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:52.793061 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:52.793147 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:52.794916 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.795645 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.797172 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.797584 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.797684 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.800454 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:52.800459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:52.800462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:52.800495 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:52.802992 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:52.803628 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:52.803632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:52.803634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:52.803765 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:52.804157 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:52.804160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:52.804162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:52.804491 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:52.804677 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:52.805142 (+ 465us) server_negotiation.cc:299] Negotiation successful
0812 07:00:52.805336 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:00:52.809877 24014 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:00:52.817059 24014 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:00:52.842767 24779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.830970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:46461 (local address 127.0.0.1:60580)
0812 07:00:52.832049 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:52.832135 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:52.832372 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:52.833242 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:52.833250 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:52.833281 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:52.833741 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:52.833754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.836139 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.836144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:52.838670 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:52.838687 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.841682 (+ 2995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.841694 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:52.841936 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.841992 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 07:00:52.842120 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 07:00:52.842343 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":3199,"threads_started":1}
I20260812 07:00:52.844254 24780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.831659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.26.226.1:46461)
0812 07:00:52.832341 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:52.832351 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:52.832386 (+ 35us) server_negotiation.cc:407] Connection header received
0812 07:00:52.832687 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:52.832699 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:52.832986 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:52.833993 (+ 1007us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:52.834299 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.835940 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.838900 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.841419 (+ 2519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.843343 (+ 1924us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.843602 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:52.843695 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 07:00:52.843877 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":1035,"threads_started":1}
I20260812 07:00:52.845784 24777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.831204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:45979 (local address 127.0.0.1:57486)
0812 07:00:52.834486 (+ 3282us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:52.834507 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:52.834634 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:52.835570 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:52.835575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:52.835601 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:52.836017 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:52.836032 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.838858 (+ 2826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.838864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:52.842248 (+ 3384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:52.842262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.844839 (+ 2577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.844850 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:52.845049 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.845088 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 07:00:52.845174 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 07:00:52.845302 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3140,"thread_start_us":1665,"threads_started":1}
I20260812 07:00:52.846935 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.832392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57486 (local address 127.26.226.3:45979)
0812 07:00:52.833272 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:52.833284 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:52.834764 (+ 1480us) server_negotiation.cc:407] Connection header received
0812 07:00:52.834868 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:52.834876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:52.835104 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:52.835302 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:52.836264 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.838682 (+ 2418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.842437 (+ 3755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.844166 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.844311 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.846434 (+ 2123us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:52.846493 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 07:00:52.846633 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":1434,"threads_started":1}
I20260812 07:00:52.850886 24207 tablet_service.cc:1467] Processing CreateTablet for tablet a219a6c51bcd4964813406ea2ddf1de2 (DEFAULT_TABLE table=test-table [id=293d5069a12d41939ece0e3b96f2a614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:00:52.850880 24637 tablet_service.cc:1467] Processing CreateTablet for tablet a219a6c51bcd4964813406ea2ddf1de2 (DEFAULT_TABLE table=test-table [id=293d5069a12d41939ece0e3b96f2a614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:00:52.852344 24207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a219a6c51bcd4964813406ea2ddf1de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:52.852370 24637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a219a6c51bcd4964813406ea2ddf1de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:52.853811 24779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.834569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33519 (local address 127.0.0.1:44554)
0812 07:00:52.843053 (+ 8484us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:52.843072 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:52.843179 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:52.846954 (+ 3775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:52.846962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:52.846994 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:52.847181 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:52.847197 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.848933 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.848940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:52.851115 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:52.851134 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.853025 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.853032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:52.853210 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.853254 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 07:00:52.853358 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 07:00:52.853505 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8368,"thread_start_us":12578,"threads_started":1}
I20260812 07:00:52.853921 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:52.834911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.26.226.2:33519)
0812 07:00:52.846414 (+ 11503us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:52.846421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:52.846447 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:00:52.846529 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:52.846535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:52.846742 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:52.846913 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:52.847357 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.848764 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.851302 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:52.852826 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:52.853002 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:52.853184 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:52.853421 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 07:00:52.853594 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11335,"thread_start_us":11187,"threads_started":1}
I20260812 07:00:52.857647 24415 tablet_service.cc:1467] Processing CreateTablet for tablet a219a6c51bcd4964813406ea2ddf1de2 (DEFAULT_TABLE table=test-table [id=293d5069a12d41939ece0e3b96f2a614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:00:52.858809 24415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a219a6c51bcd4964813406ea2ddf1de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:52.865760 24785 tablet_bootstrap.cc:492] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: Bootstrap starting.
I20260812 07:00:52.869547 24785 tablet_bootstrap.cc:654] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:52.870950 24786 tablet_bootstrap.cc:492] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: Bootstrap starting.
I20260812 07:00:52.870956 24785 log.cc:824] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:52.873910 24787 tablet_bootstrap.cc:492] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: Bootstrap starting.
I20260812 07:00:52.874788 24785 tablet_bootstrap.cc:492] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: No bootstrap required, opened a new log
I20260812 07:00:52.875088 24785 ts_tablet_manager.cc:1397] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260812 07:00:52.875088 24786 tablet_bootstrap.cc:654] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:52.876418 24786 log.cc:824] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:52.877050 24787 tablet_bootstrap.cc:654] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:52.878257 24787 log.cc:824] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:52.879663 24786 tablet_bootstrap.cc:492] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: No bootstrap required, opened a new log
I20260812 07:00:52.879923 24786 ts_tablet_manager.cc:1397] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:00:52.880874 24787 tablet_bootstrap.cc:492] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: No bootstrap required, opened a new log
I20260812 07:00:52.881153 24787 ts_tablet_manager.cc:1397] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:00:52.885914 24785 raft_consensus.cc:348] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 } } peers { permanent_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 45979 } } peers { permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 } }
I20260812 07:00:52.886386 24785 raft_consensus.cc:374] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:52.886543 24785 raft_consensus.cc:729] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 616c96aff9a949e2a3df9b114ccf1c8a, State: Initialized, Role: FOLLOWER
I20260812 07:00:52.887259 24785 consensus_queue.cc:260] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 } } peers { permanent_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 45979 } } peers { permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 } }
I20260812 07:00:52.888584 24755 heartbeater.cc:503] Master 127.26.226.62:46787 was elected leader, sending a full tablet report...
I20260812 07:00:52.889307 24785 ts_tablet_manager.cc:1428] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260812 07:00:52.890491 24787 raft_consensus.cc:348] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 } } peers { permanent_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 45979 } } peers { permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 } }
I20260812 07:00:52.890966 24787 raft_consensus.cc:374] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:52.890729 24786 raft_consensus.cc:348] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 } } peers { permanent_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 45979 } } peers { permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 } }
I20260812 07:00:52.891085 24787 raft_consensus.cc:729] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64b6d552e703413f8170ef8936cf625b, State: Initialized, Role: FOLLOWER
I20260812 07:00:52.891184 24786 raft_consensus.cc:374] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:52.891319 24786 raft_consensus.cc:729] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4294e0d0c094c1eb2a8bc9cfe457728, State: Initialized, Role: FOLLOWER
I20260812 07:00:52.891719 24787 consensus_queue.cc:260] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 } } peers { permanent_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 45979 } } peers { permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 } }
I20260812 07:00:52.891880 24786 consensus_queue.cc:260] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 } } peers { permanent_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 45979 } } peers { permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 } }
I20260812 07:00:52.893915 24786 ts_tablet_manager.cc:1428] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260812 07:00:52.895028 24517 heartbeater.cc:503] Master 127.26.226.62:46787 was elected leader, sending a full tablet report...
I20260812 07:00:52.897833 24787 ts_tablet_manager.cc:1428] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
W20260812 07:00:52.926554 24518 tablet.cc:2369] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:00:53.004599 24758 tablet.cc:2369] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:00:53.063957 24795 raft_consensus.cc:482] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:00:53.064356 24795 raft_consensus.cc:504] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 } } peers { permanent_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 45979 } } peers { permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 } }
I20260812 07:00:53.066141 24795 leader_election.cc:288] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4294e0d0c094c1eb2a8bc9cfe457728 (127.26.226.1:46461), 64b6d552e703413f8170ef8936cf625b (127.26.226.2:33519)
I20260812 07:00:53.073309 24780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:53.066694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:41975 (local address 127.26.226.1:46461)
0812 07:00:53.066941 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:53.066948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:53.066971 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:00:53.067100 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:53.067108 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:53.067209 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:53.067335 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:53.067913 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.069091 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.071120 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.072346 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.072523 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:53.072600 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:53.072865 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 07:00:53.073005 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 07:00:53.073316 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:53.066511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:46461 (local address 127.26.226.3:41975)
0812 07:00:53.066753 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:53.066779 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:53.066960 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:53.067537 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:53.067542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:53.067571 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:53.067753 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:53.067762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.069240 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.069246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:53.070950 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:53.070963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.072508 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.072514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:53.072702 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:53.072727 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:00:53.072838 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 07:00:53.072981 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":3}
I20260812 07:00:53.073925 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:53.066926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33519 (local address 127.26.226.3:38421)
0812 07:00:53.067752 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:53.067770 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:53.067911 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:53.068309 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:53.068316 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:53.068350 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:53.068714 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:53.068726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.069995 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.069998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:53.071721 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:53.071740 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.073103 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.073110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:53.073274 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:53.073298 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:00:53.073415 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 07:00:53.073601 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":463,"threads_started":1}
I20260812 07:00:53.074159 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:53.067128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:38421 (local address 127.26.226.2:33519)
0812 07:00:53.067365 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:53.067373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:53.067889 (+ 516us) server_negotiation.cc:407] Connection header received
0812 07:00:53.068051 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:53.068055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:53.068191 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:53.068258 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:53.068873 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.069842 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.071886 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.072879 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.073075 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:53.073149 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:53.073760 (+ 611us) server_negotiation.cc:299] Negotiation successful
0812 07:00:53.073894 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:00:53.074489 24238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a219a6c51bcd4964813406ea2ddf1de2" candidate_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" is_pre_election: true
I20260812 07:00:53.075093 24238 raft_consensus.cc:2454] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 616c96aff9a949e2a3df9b114ccf1c8a in term 0.
I20260812 07:00:53.075162 24444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a219a6c51bcd4964813406ea2ddf1de2" candidate_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64b6d552e703413f8170ef8936cf625b" is_pre_election: true
I20260812 07:00:53.075726 24444 raft_consensus.cc:2454] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 616c96aff9a949e2a3df9b114ccf1c8a in term 0.
I20260812 07:00:53.075760 24558 leader_election.cc:302] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 616c96aff9a949e2a3df9b114ccf1c8a, a4294e0d0c094c1eb2a8bc9cfe457728; no voters:
I20260812 07:00:53.076182 24795 raft_consensus.cc:2781] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:00:53.076370 24795 raft_consensus.cc:482] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:00:53.076503 24795 raft_consensus.cc:3037] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:53.078781 24795 raft_consensus.cc:504] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 } } peers { permanent_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 45979 } } peers { permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 } }
I20260812 07:00:53.079417 24795 leader_election.cc:288] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [CANDIDATE]: Term 1 election: Requested vote from peers a4294e0d0c094c1eb2a8bc9cfe457728 (127.26.226.1:46461), 64b6d552e703413f8170ef8936cf625b (127.26.226.2:33519)
I20260812 07:00:53.079948 24238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a219a6c51bcd4964813406ea2ddf1de2" candidate_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728"
I20260812 07:00:53.079998 24444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a219a6c51bcd4964813406ea2ddf1de2" candidate_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64b6d552e703413f8170ef8936cf625b"
I20260812 07:00:53.080214 24238 raft_consensus.cc:3037] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:53.080212 24444 raft_consensus.cc:3037] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:53.083567 24238 raft_consensus.cc:2454] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 616c96aff9a949e2a3df9b114ccf1c8a in term 1.
I20260812 07:00:53.083567 24444 raft_consensus.cc:2454] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 616c96aff9a949e2a3df9b114ccf1c8a in term 1.
I20260812 07:00:53.083971 24558 leader_election.cc:302] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 616c96aff9a949e2a3df9b114ccf1c8a, 64b6d552e703413f8170ef8936cf625b; no voters:
I20260812 07:00:53.084313 24795 raft_consensus.cc:2781] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:00:53.084824 24795 raft_consensus.cc:686] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 1 LEADER]: Becoming Leader. State: Replica: 616c96aff9a949e2a3df9b114ccf1c8a, State: Running, Role: LEADER
I20260812 07:00:53.085455 24795 consensus_queue.cc:237] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 } } peers { permanent_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 45979 } } peers { permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 } }
I20260812 07:00:53.090656 24010 catalog_manager.cc:5495] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 616c96aff9a949e2a3df9b114ccf1c8a (127.26.226.3). New cstate: current_term: 1 leader_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "616c96aff9a949e2a3df9b114ccf1c8a" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 45979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 } health_report { overall_health: UNKNOWN } } }
W20260812 07:00:53.105815 24309 tablet.cc:2369] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:00:53.141371 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:53.136286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57500 (local address 127.26.226.3:45979)
0812 07:00:53.136485 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:53.136489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:53.136523 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:00:53.136617 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:53.136621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:53.136712 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:53.136839 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:00:53.137208 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.137935 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.139375 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.139783 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.139891 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:53.140389 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:00:53.140904 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:00:53.141022 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:53.141102 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 07:00:53.141168 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:00:53.149571 24444 raft_consensus.cc:1261] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b [term 1 FOLLOWER]: Refusing update from remote peer 616c96aff9a949e2a3df9b114ccf1c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:00:53.149638 24238 raft_consensus.cc:1261] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728 [term 1 FOLLOWER]: Refusing update from remote peer 616c96aff9a949e2a3df9b114ccf1c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:00:53.150609 24804 consensus_queue.cc:1035] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b6d552e703413f8170ef8936cf625b" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:00:53.151036 24795 consensus_queue.cc:1035] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4294e0d0c094c1eb2a8bc9cfe457728" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 46461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:00:53.162674 24807 mvcc.cc:205] Tried to move back new op lower bound from 7317577945690882048 to 7317577945442324480. Current Snapshot: MvccSnapshot[applied={T|T < 7317577945690882048}]
I20260812 07:00:53.177520 24010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:00:53.177860 24010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:00:53.181420 24010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 198dc9d4535245aebc209a783547edd2: Sending DeleteTablet for 3 replicas of tablet a219a6c51bcd4964813406ea2ddf1de2
I20260812 07:00:53.182571 24207 tablet_service.cc:1514] Processing DeleteTablet for tablet a219a6c51bcd4964813406ea2ddf1de2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:00:53 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:60580
I20260812 07:00:53.182952 24637 tablet_service.cc:1514] Processing DeleteTablet for tablet a219a6c51bcd4964813406ea2ddf1de2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:00:53 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:57486
I20260812 07:00:53.183909 24415 tablet_service.cc:1514] Processing DeleteTablet for tablet a219a6c51bcd4964813406ea2ddf1de2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:00:53 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:44554
I20260812 07:00:53.190179 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 24119
I20260812 07:00:53.201332 24823 tablet_replica.cc:330] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: stopping tablet replica
I20260812 07:00:53.201838 24823 raft_consensus.cc:2229] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:00:53.202399 24823 raft_consensus.cc:2258] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:00:53.204885 24823 ts_tablet_manager.cc:1905] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:00:53.206435 24820 tablet_replica.cc:330] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: stopping tablet replica
I20260812 07:00:53.206892 24820 raft_consensus.cc:2229] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:00:53.207419 24820 raft_consensus.cc:2258] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:00:53.208423 24822 tablet_replica.cc:330] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: stopping tablet replica
I20260812 07:00:53.208861 24822 raft_consensus.cc:2229] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:00:53.209574 24822 raft_consensus.cc:2258] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:00:53.212041 24820 ts_tablet_manager.cc:1905] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:00:53.214848 24823 ts_tablet_manager.cc:1918] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:00:53.215039 24823 log.cc:1196] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/a219a6c51bcd4964813406ea2ddf1de2
I20260812 07:00:53.215644 24823 ts_tablet_manager.cc:1939] T a219a6c51bcd4964813406ea2ddf1de2 P 64b6d552e703413f8170ef8936cf625b: Deleting consensus metadata
I20260812 07:00:53.217790 23983 catalog_manager.cc:4841] TS 64b6d552e703413f8170ef8936cf625b (127.26.226.2:33519): tablet a219a6c51bcd4964813406ea2ddf1de2 (table test-table [id=293d5069a12d41939ece0e3b96f2a614]) successfully deleted
I20260812 07:00:53.218577 24822 ts_tablet_manager.cc:1905] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:00:53.219092 24820 ts_tablet_manager.cc:1918] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 12 07:00:53 dist-test-slave-t3q3 krb5kdc[23930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 07:00:53.219245 24820 log.cc:1196] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/a219a6c51bcd4964813406ea2ddf1de2
I20260812 07:00:53.219775 24820 ts_tablet_manager.cc:1939] T a219a6c51bcd4964813406ea2ddf1de2 P a4294e0d0c094c1eb2a8bc9cfe457728: Deleting consensus metadata
I20260812 07:00:53.221891 23983 catalog_manager.cc:4841] TS a4294e0d0c094c1eb2a8bc9cfe457728 (127.26.226.1:46461): tablet a219a6c51bcd4964813406ea2ddf1de2 (table test-table [id=293d5069a12d41939ece0e3b96f2a614]) successfully deleted
I20260812 07:00:53.226377 24822 ts_tablet_manager.cc:1918] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:00:53.226583 24822 log.cc:1196] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/a219a6c51bcd4964813406ea2ddf1de2
I20260812 07:00:53.227217 24822 ts_tablet_manager.cc:1939] T a219a6c51bcd4964813406ea2ddf1de2 P 616c96aff9a949e2a3df9b114ccf1c8a: Deleting consensus metadata
I20260812 07:00:53.228994 24780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:53.191583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60594 (local address 127.26.226.1:46461)
0812 07:00:53.191824 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:53.191833 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:53.200663 (+ 8830us) server_negotiation.cc:407] Connection header received
0812 07:00:53.200876 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:53.200885 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:53.201004 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:53.201158 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:53.207892 (+ 6734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.209071 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.211484 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.212249 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.212356 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:53.220784 (+ 8428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:53.220803 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:53.220823 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:53.220879 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:53.225182 (+ 4303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:53.226129 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:53.226142 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:53.226145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:53.226332 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:53.226892 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:53.226900 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:53.226906 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:53.227395 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:53.227701 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:53.228142 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 07:00:53.228391 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 07:00:53.229158 23978 catalog_manager.cc:4841] TS 616c96aff9a949e2a3df9b114ccf1c8a (127.26.226.3:45979): tablet a219a6c51bcd4964813406ea2ddf1de2 (table test-table [id=293d5069a12d41939ece0e3b96f2a614]) successfully deleted
I20260812 07:00:53.229677 24297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:53.323390 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 24119
I20260812 07:00:53.350850 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 24320
Aug 12 07:00:53 dist-test-slave-t3q3 krb5kdc[23930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 07:00:53.369879 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:53.351738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44566 (local address 127.26.226.2:33519)
0812 07:00:53.351941 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:53.351946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:53.352111 (+ 165us) server_negotiation.cc:407] Connection header received
0812 07:00:53.352296 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:53.352300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:53.352364 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:53.352449 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:53.354170 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.354911 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.357016 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.357385 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.357482 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:53.362697 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:53.362717 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:53.362741 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:53.362792 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:53.366743 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:53.367566 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:53.367577 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:53.367581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:53.367751 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:53.368152 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:53.368158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:53.368161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:53.368702 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:53.369024 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:53.369248 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 07:00:53.369489 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:00:53.370630 24506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:53.463303 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 24320
I20260812 07:00:53.490377 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 24528
Aug 12 07:00:53 dist-test-slave-t3q3 krb5kdc[23930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 07:00:53.510895 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:53.491228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57502 (local address 127.26.226.3:45979)
0812 07:00:53.491410 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:53.491415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:53.491433 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:00:53.491662 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:53.491667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:53.491791 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:53.491885 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:53.493973 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.494850 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.497113 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:53.497589 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:53.497763 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:53.503176 (+ 5413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:53.503201 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:53.503224 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:53.503281 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:53.507510 (+ 4229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:53.508445 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:53.508456 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:53.508460 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:53.508685 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:53.509118 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:53.509124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:53.509127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:53.509619 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:53.510000 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:53.510189 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 07:00:53.510439 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 07:00:53.511746 24746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:53.635367 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 24528
I20260812 07:00:53.664342 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 23948
I20260812 07:00:53.665642 24057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:53.778586 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 23948
2026-08-12T07:00:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5044 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T07:00:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:00:53Z Disabled control of system clock
I20260812 07:00:53.888926 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:35475
--webserver_interface=127.26.226.62
--webserver_port=0
--builtin_ntp_servers=127.26.226.20:40137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:35475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:00:54.244729 24837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:54.245033 24837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:54.245198 24837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:54.252595 24837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:00:54.252710 24837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:54.252812 24837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:54.252920 24837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:00:54.253021 24837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:00:54.262418 24837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:40137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:35475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:35475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:54.263857 24837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:54.265760 24837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:54.278172 24847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:54.278143 24845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:54.278146 24844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:54.279050 24837 server_base.cc:1034] running on GCE node
I20260812 07:00:54.279834 24837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:54.281337 24837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:54.282562 24837 hybrid_clock.cc:648] HybridClock initialized: now 1786518054282518 us; error 37 us; skew 500 ppm
I20260812 07:00:54.282960 24837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:00:54.285364 24837 webserver.cc:460] Webserver started at http://127.26.226.62:35133/ using document root <none> and password file <none>
I20260812 07:00:54.286079 24837 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:54.286198 24837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:54.286567 24837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:00:54.289578 24837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "cf8a78a7aa3d4649a522488ab7a81202"
format_stamp: "Formatted at 2026-08-12 07:00:54 on dist-test-slave-t3q3"
server_key: "066af2394909aa89d0cc609cc638f3b8"
server_key_iv: "2f09eab73cd419b1dd0f588d3901ffe5"
server_key_version: "encryptionkey@0"
I20260812 07:00:54.290421 24837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "cf8a78a7aa3d4649a522488ab7a81202"
format_stamp: "Formatted at 2026-08-12 07:00:54 on dist-test-slave-t3q3"
server_key: "066af2394909aa89d0cc609cc638f3b8"
server_key_iv: "2f09eab73cd419b1dd0f588d3901ffe5"
server_key_version: "encryptionkey@0"
I20260812 07:00:54.295612 24837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:00:54.299429 24858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:54.300872 24837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:00:54.301079 24837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "cf8a78a7aa3d4649a522488ab7a81202"
format_stamp: "Formatted at 2026-08-12 07:00:54 on dist-test-slave-t3q3"
server_key: "066af2394909aa89d0cc609cc638f3b8"
server_key_iv: "2f09eab73cd419b1dd0f588d3901ffe5"
server_key_version: "encryptionkey@0"
I20260812 07:00:54.301311 24837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:54.341351 24837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:54.342317 24837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:54.342645 24837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:54.362425 24837 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:35475
I20260812 07:00:54.362397 24945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:35475 every 8 connection(s)
I20260812 07:00:54.363864 24837 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:00:54.367753 24946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:54.372836 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 24837
I20260812 07:00:54.373059 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:00:54.373572 27528 external_mini_cluster.cc:1428] Setting key 2c40d813632380a3fae64ab6ec12d992
I20260812 07:00:54.378942 24946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202: Bootstrap starting.
I20260812 07:00:54.382691 24946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:54.383919 24946 log.cc:824] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:54.387189 24946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202: No bootstrap required, opened a new log
I20260812 07:00:54.388000 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:54.375587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47228 (local address 127.26.226.62:35475)
0812 07:00:54.376805 (+ 1218us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:54.376816 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:54.376859 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:00:54.378548 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:54.378571 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:54.379004 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:54.379373 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:54.380021 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:54.381399 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:54.383383 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:54.384165 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:54.384307 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:54.385722 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:54.385730 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:54.385744 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:00:54.386294 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:54.386367 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:00:54.386388 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:54.386475 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:54.386744 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 07:00:54.387049 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":522,"threads_started":1}
I20260812 07:00:54.394805 24946 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf8a78a7aa3d4649a522488ab7a81202" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35475 } }
I20260812 07:00:54.395232 24946 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:54.395360 24946 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf8a78a7aa3d4649a522488ab7a81202, State: Initialized, Role: FOLLOWER
I20260812 07:00:54.395933 24946 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf8a78a7aa3d4649a522488ab7a81202" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35475 } }
I20260812 07:00:54.396226 24946 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:00:54.396384 24946 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:00:54.396590 24946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:54.399024 24946 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf8a78a7aa3d4649a522488ab7a81202" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35475 } }
I20260812 07:00:54.399547 24946 leader_election.cc:302] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf8a78a7aa3d4649a522488ab7a81202; no voters:
I20260812 07:00:54.400202 24946 leader_election.cc:288] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:00:54.400352 24953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:00:54.401104 24953 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [term 1 LEADER]: Becoming Leader. State: Replica: cf8a78a7aa3d4649a522488ab7a81202, State: Running, Role: LEADER
I20260812 07:00:54.401675 24953 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf8a78a7aa3d4649a522488ab7a81202" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35475 } }
I20260812 07:00:54.402755 24946 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:00:54.405141 24955 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf8a78a7aa3d4649a522488ab7a81202. Latest consensus state: current_term: 1 leader_uuid: "cf8a78a7aa3d4649a522488ab7a81202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf8a78a7aa3d4649a522488ab7a81202" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35475 } } }
I20260812 07:00:54.405141 24954 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf8a78a7aa3d4649a522488ab7a81202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf8a78a7aa3d4649a522488ab7a81202" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35475 } } }
I20260812 07:00:54.405696 24955 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [sys.catalog]: This master's current role is: LEADER
I20260812 07:00:54.405761 24954 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202 [sys.catalog]: This master's current role is: LEADER
I20260812 07:00:54.413025 24958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:00:54.418689 24958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:00:54.428280 24958 catalog_manager.cc:1347] Generated new cluster ID: 724632c6fee844d8aba48e9e6d2f8516
I20260812 07:00:54.428404 24958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:00:54.456133 24958 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:00:54.457454 24958 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:00:54.467798 24958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202: Generated new TSK 0
I20260812 07:00:54.468761 24958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:00:54.479467 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35475
--builtin_ntp_servers=127.26.226.20:40137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:00:54.882387 24987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:54.882664 24987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:54.882751 24987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:54.890264 24987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:54.890372 24987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:54.890480 24987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 07:00:54.900483 24987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:40137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:54.901770 24987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:54.903739 24987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:54.917683 24993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:54.917634 24992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:54.917730 24996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:54.918339 24987 server_base.cc:1034] running on GCE node
I20260812 07:00:54.918954 24987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:54.919919 24987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:54.921155 24987 hybrid_clock.cc:648] HybridClock initialized: now 1786518054921102 us; error 46 us; skew 500 ppm
I20260812 07:00:54.921473 24987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:00:54.923878 24987 webserver.cc:460] Webserver started at http://127.26.226.1:43667/ using document root <none> and password file <none>
I20260812 07:00:54.924669 24987 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:54.924809 24987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:54.925143 24987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:00:54.928143 24987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "36953873942542b99f3b1a08272e2fac"
format_stamp: "Formatted at 2026-08-12 07:00:54 on dist-test-slave-t3q3"
server_key: "39187010b72913afc9f4552fb3c41ef0"
server_key_iv: "db254145cca915f0181fa60bdf7d36dc"
server_key_version: "encryptionkey@0"
I20260812 07:00:54.928936 24987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "36953873942542b99f3b1a08272e2fac"
format_stamp: "Formatted at 2026-08-12 07:00:54 on dist-test-slave-t3q3"
server_key: "39187010b72913afc9f4552fb3c41ef0"
server_key_iv: "db254145cca915f0181fa60bdf7d36dc"
server_key_version: "encryptionkey@0"
I20260812 07:00:54.933948 24987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:00:54.937388 25007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:54.938740 24987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:00:54.938911 24987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "36953873942542b99f3b1a08272e2fac"
format_stamp: "Formatted at 2026-08-12 07:00:54 on dist-test-slave-t3q3"
server_key: "39187010b72913afc9f4552fb3c41ef0"
server_key_iv: "db254145cca915f0181fa60bdf7d36dc"
server_key_version: "encryptionkey@0"
I20260812 07:00:54.939123 24987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:54.956938 24987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:54.957767 24987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:54.958083 24987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:54.959872 24987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:54.961309 24987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:54.961401 24987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:54.961480 24987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:54.961552 24987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:54.994585 24987 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:32831
I20260812 07:00:54.994637 25172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:32831 every 8 connection(s)
I20260812 07:00:54.995970 24987 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 07:00:54.999128 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 24987
I20260812 07:00:54.999343 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 07:00:54.999822 27528 external_mini_cluster.cc:1428] Setting key 13325a3a9d033985e3de7f0599ee34da
I20260812 07:00:55.002929 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35475
--builtin_ntp_servers=127.26.226.20:40137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:00:55.010792 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:54.998673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:56967 (local address 127.26.226.62:35475)
0812 07:00:54.999228 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:54.999236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:55.000920 (+ 1684us) server_negotiation.cc:407] Connection header received
0812 07:00:55.002353 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:55.002360 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:55.002449 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:55.002587 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:55.003783 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.005059 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.007662 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.008125 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.008236 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:55.009860 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:55.009865 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:55.009868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:00:55.009886 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:55.009935 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:00:55.009947 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:55.010020 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:55.010286 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 07:00:55.010393 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":266,"threads_started":1}
I20260812 07:00:55.012135 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:54.999317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:35475 (local address 127.26.226.1:56967)
0812 07:00:55.000446 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:55.000491 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:55.002096 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:55.002733 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:55.002741 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:55.002929 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:00:55.003602 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:55.003619 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.005275 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.005284 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:55.007499 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:55.007518 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.008361 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.008371 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:55.008719 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:55.009480 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:00:55.009485 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:00:55.009539 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:00:55.009650 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:00:55.010050 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:00:55.010063 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:00:55.010074 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:00:55.010272 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 07:00:55.010713 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":358,"threads_started":1}
I20260812 07:00:55.013716 25175 heartbeater.cc:347] Connected to a master server at 127.26.226.62:35475
I20260812 07:00:55.014012 25175 heartbeater.cc:464] Registering TS with master...
I20260812 07:00:55.014770 25175 heartbeater.cc:511] Master 127.26.226.62:35475 requested a full tablet report, sending...
I20260812 07:00:55.016855 24886 ts_manager.cc:194] Registered new tserver with Master: 36953873942542b99f3b1a08272e2fac (127.26.226.1:32831)
I20260812 07:00:55.018302 24886 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.1:56967
W20260812 07:00:55.359750 25184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:55.360047 25184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:55.360216 25184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:55.367587 25184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:55.367740 25184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:55.367923 25184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 07:00:55.377079 25184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:40137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:55.378459 25184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:55.380295 25184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:55.396198 25195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:55.396234 25197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:55.396260 25194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:55.397184 25184 server_base.cc:1034] running on GCE node
I20260812 07:00:55.397828 25184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:55.398841 25184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:55.400192 25184 hybrid_clock.cc:648] HybridClock initialized: now 1786518055400143 us; error 48 us; skew 500 ppm
I20260812 07:00:55.400545 25184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:00:55.403067 25184 webserver.cc:460] Webserver started at http://127.26.226.2:33613/ using document root <none> and password file <none>
I20260812 07:00:55.403873 25184 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:55.404006 25184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:55.404374 25184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:00:55.407424 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "77ba4bf54855463689bf0d4a67d50abb"
format_stamp: "Formatted at 2026-08-12 07:00:55 on dist-test-slave-t3q3"
server_key: "a3543f095ac332a31b7564ca7c45f32e"
server_key_iv: "183a75f4b92a598348ead34f0f47343e"
server_key_version: "encryptionkey@0"
I20260812 07:00:55.408315 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "77ba4bf54855463689bf0d4a67d50abb"
format_stamp: "Formatted at 2026-08-12 07:00:55 on dist-test-slave-t3q3"
server_key: "a3543f095ac332a31b7564ca7c45f32e"
server_key_iv: "183a75f4b92a598348ead34f0f47343e"
server_key_version: "encryptionkey@0"
I20260812 07:00:55.413484 25184 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260812 07:00:55.416917 25206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:55.418184 25184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:00:55.418367 25184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "77ba4bf54855463689bf0d4a67d50abb"
format_stamp: "Formatted at 2026-08-12 07:00:55 on dist-test-slave-t3q3"
server_key: "a3543f095ac332a31b7564ca7c45f32e"
server_key_iv: "183a75f4b92a598348ead34f0f47343e"
server_key_version: "encryptionkey@0"
I20260812 07:00:55.418586 25184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:55.434944 25184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:55.435786 25184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:55.436146 25184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:55.437145 25184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:55.438598 25184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:55.438731 25184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:55.438856 25184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:55.438943 25184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:55.473909 25184 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33867
I20260812 07:00:55.473964 25385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33867 every 8 connection(s)
I20260812 07:00:55.475287 25184 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 07:00:55.479818 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 25184
I20260812 07:00:55.480046 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 07:00:55.480701 27528 external_mini_cluster.cc:1428] Setting key 897e152370e91889315f4ee0566fd904
I20260812 07:00:55.484226 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35475
--builtin_ntp_servers=127.26.226.20:40137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:00:55.490674 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:55.478011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:60095 (local address 127.26.226.62:35475)
0812 07:00:55.478203 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:55.478214 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:55.480051 (+ 1837us) server_negotiation.cc:407] Connection header received
0812 07:00:55.482404 (+ 2353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:55.482409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:55.482509 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:55.482631 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:55.484168 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.484970 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.487555 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.487972 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.488146 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:55.489884 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:55.489888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:55.489892 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:00:55.489912 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:55.489947 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:00:55.489960 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:55.490043 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:55.490323 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 07:00:55.490410 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:00:55.492312 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:55.478581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:35475 (local address 127.26.226.2:60095)
0812 07:00:55.479832 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:55.479883 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:55.482103 (+ 2220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:55.482887 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:55.482897 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:55.483047 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:00:55.483916 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:55.483946 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.485152 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.485161 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:55.487378 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:55.487398 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.488256 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.488266 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:55.488631 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:55.489470 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:00:55.489475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:00:55.489542 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:00:55.489660 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:00:55.490068 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:00:55.490082 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:00:55.490093 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:00:55.490303 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 07:00:55.490763 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":381,"threads_started":1}
I20260812 07:00:55.494043 25386 heartbeater.cc:347] Connected to a master server at 127.26.226.62:35475
I20260812 07:00:55.494416 25386 heartbeater.cc:464] Registering TS with master...
I20260812 07:00:55.495426 25386 heartbeater.cc:511] Master 127.26.226.62:35475 requested a full tablet report, sending...
I20260812 07:00:55.497125 24886 ts_manager.cc:194] Registered new tserver with Master: 77ba4bf54855463689bf0d4a67d50abb (127.26.226.2:33867)
I20260812 07:00:55.498020 24886 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.2:60095
W20260812 07:00:55.839740 25391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:55.840039 25391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:55.840255 25391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:55.847698 25391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:55.847865 25391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:55.848030 25391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 07:00:55.857394 25391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:40137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:55.858740 25391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:55.860625 25391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:55.873942 25396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:55.873850 25398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:55.874018 25401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:55.875002 25391 server_base.cc:1034] running on GCE node
I20260812 07:00:55.875670 25391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:55.876732 25391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:55.878000 25391 hybrid_clock.cc:648] HybridClock initialized: now 1786518055877951 us; error 44 us; skew 500 ppm
I20260812 07:00:55.878352 25391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:00:55.880662 25391 webserver.cc:460] Webserver started at http://127.26.226.3:41083/ using document root <none> and password file <none>
I20260812 07:00:55.881389 25391 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:55.881529 25391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:55.881856 25391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:00:55.884811 25391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "3fd1ed5419154459afef529ff6c0a071"
format_stamp: "Formatted at 2026-08-12 07:00:55 on dist-test-slave-t3q3"
server_key: "ffa1c8f568b004f3ce8c1306af4c398f"
server_key_iv: "d39224ea675d6484c5de0a71dce260a4"
server_key_version: "encryptionkey@0"
I20260812 07:00:55.885634 25391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "3fd1ed5419154459afef529ff6c0a071"
format_stamp: "Formatted at 2026-08-12 07:00:55 on dist-test-slave-t3q3"
server_key: "ffa1c8f568b004f3ce8c1306af4c398f"
server_key_iv: "d39224ea675d6484c5de0a71dce260a4"
server_key_version: "encryptionkey@0"
I20260812 07:00:55.890843 25391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:00:55.894778 25410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:55.896165 25391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:00:55.896353 25391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "3fd1ed5419154459afef529ff6c0a071"
format_stamp: "Formatted at 2026-08-12 07:00:55 on dist-test-slave-t3q3"
server_key: "ffa1c8f568b004f3ce8c1306af4c398f"
server_key_iv: "d39224ea675d6484c5de0a71dce260a4"
server_key_version: "encryptionkey@0"
I20260812 07:00:55.896589 25391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:55.935609 25391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:55.936547 25391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:55.936864 25391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:55.937893 25391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:55.939445 25391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:55.939576 25391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:55.939721 25391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:55.939814 25391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:55.975306 25391 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:39005
I20260812 07:00:55.975399 25584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:39005 every 8 connection(s)
I20260812 07:00:55.976751 25391 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 07:00:55.980580 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 25391
I20260812 07:00:55.980784 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 07:00:55.981393 27528 external_mini_cluster.cc:1428] Setting key d58be2df429a2ed9e4a6392c856613a5
I20260812 07:00:55.989698 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:55.979465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:49175 (local address 127.26.226.62:35475)
0812 07:00:55.979693 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:55.979699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:55.981305 (+ 1606us) server_negotiation.cc:407] Connection header received
0812 07:00:55.983293 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:55.983298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:55.983396 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:55.983510 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:55.984638 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.985356 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.986897 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.987309 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.987450 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:55.988844 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:55.988850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:55.988853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:00:55.988870 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:55.988901 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:00:55.988913 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:55.989006 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:55.989253 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 07:00:55.989353 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 07:00:55.990648 25589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:55.979995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:35475 (local address 127.26.226.3:49175)
0812 07:00:55.981132 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:55.981178 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:55.983018 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:55.983748 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:55.983756 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:55.983846 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:00:55.984465 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:55.984482 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.985533 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.985537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:55.986745 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:55.986757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:55.987539 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:55.987546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:55.987787 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:55.988470 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:00:55.988472 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:00:55.988510 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:00:55.988600 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:00:55.989023 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:00:55.989035 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:00:55.989045 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 07:00:55.989222 (+ 177us) client_negotiation.cc:240] Negotiation successful
0812 07:00:55.989617 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":338,"threads_started":1}
I20260812 07:00:55.991950 25586 heartbeater.cc:347] Connected to a master server at 127.26.226.62:35475
I20260812 07:00:55.992298 25586 heartbeater.cc:464] Registering TS with master...
I20260812 07:00:55.993243 25586 heartbeater.cc:511] Master 127.26.226.62:35475 requested a full tablet report, sending...
I20260812 07:00:55.994684 24886 ts_manager.cc:194] Registered new tserver with Master: 3fd1ed5419154459afef529ff6c0a071 (127.26.226.3:39005)
I20260812 07:00:55.995802 24886 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.3:49175
I20260812 07:00:55.996845 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:00:56.008502 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.001862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.26.226.62:35475)
0812 07:00:56.002095 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:56.002105 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:56.002543 (+ 438us) server_negotiation.cc:407] Connection header received
0812 07:00:56.002754 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:56.002760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:56.002830 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:56.002938 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:56.003458 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.004326 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.006004 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.006464 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.006565 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.007689 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:56.007694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:56.007698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:00:56.007721 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:56.007755 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:00:56.007767 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:56.007853 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:56.008047 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 07:00:56.008233 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:00:56.012873 24886 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:00:56.020257 24886 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:00:56.021733 25175 heartbeater.cc:503] Master 127.26.226.62:35475 was elected leader, sending a full tablet report...
I20260812 07:00:56.056928 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.034513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40012 (local address 127.26.226.3:39005)
0812 07:00:56.047792 (+ 13279us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:56.047803 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:56.047840 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:00:56.047949 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:56.047957 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:56.048261 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:56.048457 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:56.050562 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.052083 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.054782 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.055784 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.055931 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.056234 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:56.056314 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 07:00:56.056497 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13087,"thread_start_us":12932,"threads_started":1}
I20260812 07:00:56.057039 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.034236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33867 (local address 127.0.0.1:42100)
0812 07:00:56.047060 (+ 12824us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.047083 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:56.047759 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:56.048775 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:56.048783 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:56.048830 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:56.049323 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:56.049340 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.051372 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.051377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:56.053498 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:56.053513 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.055780 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.055784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:56.056014 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.056111 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 07:00:56.056247 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 07:00:56.056495 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12653,"mutex_wait_us":31,"spinlock_wait_cycles":3840,"thread_start_us":12458,"threads_started":1}
I20260812 07:00:56.057754 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.034416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42100 (local address 127.26.226.2:33867)
0812 07:00:56.044810 (+ 10394us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:56.044825 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:56.047217 (+ 2392us) server_negotiation.cc:407] Connection header received
0812 07:00:56.047986 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:56.047994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:56.048271 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:56.048454 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:56.049583 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.051205 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.053770 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.055267 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.055380 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.055666 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:56.057309 (+ 1643us) server_negotiation.cc:299] Negotiation successful
0812 07:00:56.057449 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10116,"thread_start_us":10419,"threads_started":1}
I20260812 07:00:56.057062 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.034046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:39005 (local address 127.0.0.1:40012)
0812 07:00:56.045942 (+ 11896us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.046006 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:56.046301 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:56.048775 (+ 2474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:56.048783 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:56.048826 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:56.050330 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:56.050343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.052269 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.052275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:56.054541 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:56.054557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.055984 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.055990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:56.056166 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.056203 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 07:00:56.056271 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 07:00:56.056495 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11420,"mutex_wait_us":134,"spinlock_wait_cycles":4608,"thread_start_us":11238,"threads_started":1}
I20260812 07:00:56.059216 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.047492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.26.226.1:32831)
0812 07:00:56.049310 (+ 1818us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:56.049319 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:56.050130 (+ 811us) server_negotiation.cc:407] Connection header received
0812 07:00:56.050259 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:56.050267 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:56.050469 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:56.050668 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:56.051576 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.053054 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.055700 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.057429 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.057580 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.057847 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:56.058290 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 07:00:56.058733 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1614,"thread_start_us":1451,"threads_started":1}
I20260812 07:00:56.059535 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.047655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:32831 (local address 127.0.0.1:57308)
0812 07:00:56.049782 (+ 2127us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.049802 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:56.049965 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:56.050924 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:56.050928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:56.050952 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:56.051376 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:56.051397 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.053222 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.053227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:56.055481 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:56.055496 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.057661 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.057668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:56.057858 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.057903 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 07:00:56.058017 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 07:00:56.059166 (+ 1149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2010,"mutex_wait_us":1,"spinlock_wait_cycles":1280,"thread_start_us":1866,"threads_started":1}
I20260812 07:00:56.062165 25271 tablet_service.cc:1467] Processing CreateTablet for tablet 414ef0dd0ced48e2ad0261af51f4871c (DEFAULT_TABLE table=test-table [id=4193351e124a4ab580c934bf469b9b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:00:56.063416 25271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 414ef0dd0ced48e2ad0261af51f4871c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:56.063293 25078 tablet_service.cc:1467] Processing CreateTablet for tablet 414ef0dd0ced48e2ad0261af51f4871c (DEFAULT_TABLE table=test-table [id=4193351e124a4ab580c934bf469b9b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:00:56.063293 25481 tablet_service.cc:1467] Processing CreateTablet for tablet 414ef0dd0ced48e2ad0261af51f4871c (DEFAULT_TABLE table=test-table [id=4193351e124a4ab580c934bf469b9b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:00:56.064854 25481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 414ef0dd0ced48e2ad0261af51f4871c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:56.064965 25078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 414ef0dd0ced48e2ad0261af51f4871c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:00:56.075623 25613 tablet_bootstrap.cc:492] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: Bootstrap starting.
I20260812 07:00:56.079137 25613 tablet_bootstrap.cc:654] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:56.080325 25613 log.cc:824] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:56.080912 25614 tablet_bootstrap.cc:492] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac: Bootstrap starting.
I20260812 07:00:56.083102 25615 tablet_bootstrap.cc:492] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: Bootstrap starting.
I20260812 07:00:56.084800 25613 tablet_bootstrap.cc:492] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: No bootstrap required, opened a new log
I20260812 07:00:56.084954 25614 tablet_bootstrap.cc:654] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:56.085146 25613 ts_tablet_manager.cc:1397] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260812 07:00:56.086279 25614 log.cc:824] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:56.086870 25615 tablet_bootstrap.cc:654] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:56.088325 25615 log.cc:824] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:56.090252 25614 tablet_bootstrap.cc:492] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac: No bootstrap required, opened a new log
I20260812 07:00:56.090552 25614 ts_tablet_manager.cc:1397] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 07:00:56.091488 25615 tablet_bootstrap.cc:492] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: No bootstrap required, opened a new log
I20260812 07:00:56.091759 25615 ts_tablet_manager.cc:1397] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:00:56.093724 25613 raft_consensus.cc:348] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 } } peers { permanent_uuid: "77ba4bf54855463689bf0d4a67d50abb" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33867 } } peers { permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 } }
I20260812 07:00:56.094105 25613 raft_consensus.cc:374] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:56.094241 25613 raft_consensus.cc:729] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ba4bf54855463689bf0d4a67d50abb, State: Initialized, Role: FOLLOWER
I20260812 07:00:56.094869 25613 consensus_queue.cc:260] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 } } peers { permanent_uuid: "77ba4bf54855463689bf0d4a67d50abb" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33867 } } peers { permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 } }
I20260812 07:00:56.095954 25386 heartbeater.cc:503] Master 127.26.226.62:35475 was elected leader, sending a full tablet report...
I20260812 07:00:56.097927 25613 ts_tablet_manager.cc:1428] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260812 07:00:56.101226 25614 raft_consensus.cc:348] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 } } peers { permanent_uuid: "77ba4bf54855463689bf0d4a67d50abb" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33867 } } peers { permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 } }
I20260812 07:00:56.101722 25614 raft_consensus.cc:374] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:56.101840 25614 raft_consensus.cc:729] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36953873942542b99f3b1a08272e2fac, State: Initialized, Role: FOLLOWER
I20260812 07:00:56.102491 25614 consensus_queue.cc:260] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 } } peers { permanent_uuid: "77ba4bf54855463689bf0d4a67d50abb" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33867 } } peers { permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 } }
I20260812 07:00:56.104455 25614 ts_tablet_manager.cc:1428] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260812 07:00:56.106469 25615 raft_consensus.cc:348] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 } } peers { permanent_uuid: "77ba4bf54855463689bf0d4a67d50abb" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33867 } } peers { permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 } }
I20260812 07:00:56.106842 25615 raft_consensus.cc:374] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:56.106957 25615 raft_consensus.cc:729] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fd1ed5419154459afef529ff6c0a071, State: Initialized, Role: FOLLOWER
I20260812 07:00:56.107479 25615 consensus_queue.cc:260] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 } } peers { permanent_uuid: "77ba4bf54855463689bf0d4a67d50abb" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33867 } } peers { permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 } }
I20260812 07:00:56.108592 25586 heartbeater.cc:503] Master 127.26.226.62:35475 was elected leader, sending a full tablet report...
I20260812 07:00:56.109126 25615 ts_tablet_manager.cc:1428] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
W20260812 07:00:56.227469 25388 tablet.cc:2369] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:00:56.228577 25587 tablet.cc:2369] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:00:56.248513 25177 tablet.cc:2369] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:00:56.287792 25620 raft_consensus.cc:482] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:00:56.288209 25620 raft_consensus.cc:504] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 } } peers { permanent_uuid: "77ba4bf54855463689bf0d4a67d50abb" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33867 } } peers { permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 } }
I20260812 07:00:56.290132 25620 leader_election.cc:288] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fd1ed5419154459afef529ff6c0a071 (127.26.226.3:39005), 36953873942542b99f3b1a08272e2fac (127.26.226.1:32831)
I20260812 07:00:56.297538 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.290829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:57869 (local address 127.26.226.1:32831)
0812 07:00:56.291044 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:56.291050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:56.291559 (+ 509us) server_negotiation.cc:407] Connection header received
0812 07:00:56.291737 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:56.291741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:56.291858 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:56.291965 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:56.292722 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.293428 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.295651 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.296520 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.296646 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.296696 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:56.297151 (+ 455us) server_negotiation.cc:299] Negotiation successful
0812 07:00:56.297280 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 07:00:56.297587 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.290502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:54157 (local address 127.26.226.3:39005)
0812 07:00:56.290806 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:56.290816 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:56.291269 (+ 453us) server_negotiation.cc:407] Connection header received
0812 07:00:56.291583 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:56.291589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:56.291685 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:56.291780 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:00:56.292575 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.293417 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.295704 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.296679 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.296870 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.296942 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:56.297187 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 07:00:56.297323 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 07:00:56.297683 25626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.290555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:32831 (local address 127.26.226.2:57869)
0812 07:00:56.291411 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.291427 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:56.291596 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:56.292011 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:56.292015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:56.292047 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:56.292533 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:56.292548 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.293629 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.293634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:56.295428 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:56.295442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.296732 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.296742 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:56.296949 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.296999 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 07:00:56.297088 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 07:00:56.297250 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":6,"thread_start_us":487,"threads_started":1}
I20260812 07:00:56.297709 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.290270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:39005 (local address 127.26.226.2:54157)
0812 07:00:56.291126 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.291150 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:56.291390 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:56.291824 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:56.291829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:56.291855 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:00:56.292380 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:56.292393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.293563 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.293571 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:56.295429 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:56.295442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.296875 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.296880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:56.297001 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.297024 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:00:56.297139 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 07:00:56.297278 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"mutex_wait_us":166,"spinlock_wait_cycles":93568,"thread_start_us":768,"threads_started":1}
I20260812 07:00:56.298986 25512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "414ef0dd0ced48e2ad0261af51f4871c" candidate_uuid: "77ba4bf54855463689bf0d4a67d50abb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fd1ed5419154459afef529ff6c0a071" is_pre_election: true
I20260812 07:00:56.299167 25105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "414ef0dd0ced48e2ad0261af51f4871c" candidate_uuid: "77ba4bf54855463689bf0d4a67d50abb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36953873942542b99f3b1a08272e2fac" is_pre_election: true
I20260812 07:00:56.299666 25512 raft_consensus.cc:2454] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77ba4bf54855463689bf0d4a67d50abb in term 0.
I20260812 07:00:56.299799 25105 raft_consensus.cc:2454] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77ba4bf54855463689bf0d4a67d50abb in term 0.
I20260812 07:00:56.300482 25207 leader_election.cc:302] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 36953873942542b99f3b1a08272e2fac, 3fd1ed5419154459afef529ff6c0a071, 77ba4bf54855463689bf0d4a67d50abb; no voters:
I20260812 07:00:56.300946 25620 raft_consensus.cc:2781] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:00:56.301110 25620 raft_consensus.cc:482] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:00:56.301249 25620 raft_consensus.cc:3037] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:56.303638 25620 raft_consensus.cc:504] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 } } peers { permanent_uuid: "77ba4bf54855463689bf0d4a67d50abb" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33867 } } peers { permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 } }
I20260812 07:00:56.304399 25620 leader_election.cc:288] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [CANDIDATE]: Term 1 election: Requested vote from peers 3fd1ed5419154459afef529ff6c0a071 (127.26.226.3:39005), 36953873942542b99f3b1a08272e2fac (127.26.226.1:32831)
I20260812 07:00:56.304788 25512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "414ef0dd0ced48e2ad0261af51f4871c" candidate_uuid: "77ba4bf54855463689bf0d4a67d50abb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fd1ed5419154459afef529ff6c0a071"
I20260812 07:00:56.304952 25105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "414ef0dd0ced48e2ad0261af51f4871c" candidate_uuid: "77ba4bf54855463689bf0d4a67d50abb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36953873942542b99f3b1a08272e2fac"
I20260812 07:00:56.305012 25512 raft_consensus.cc:3037] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:56.305190 25105 raft_consensus.cc:3037] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:56.307426 25512 raft_consensus.cc:2454] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77ba4bf54855463689bf0d4a67d50abb in term 1.
I20260812 07:00:56.307706 25105 raft_consensus.cc:2454] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77ba4bf54855463689bf0d4a67d50abb in term 1.
I20260812 07:00:56.307909 25209 leader_election.cc:302] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fd1ed5419154459afef529ff6c0a071, 77ba4bf54855463689bf0d4a67d50abb; no voters:
I20260812 07:00:56.308312 25620 raft_consensus.cc:2781] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:00:56.308838 25620 raft_consensus.cc:686] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 1 LEADER]: Becoming Leader. State: Replica: 77ba4bf54855463689bf0d4a67d50abb, State: Running, Role: LEADER
I20260812 07:00:56.309340 25620 consensus_queue.cc:237] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 } } peers { permanent_uuid: "77ba4bf54855463689bf0d4a67d50abb" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33867 } } peers { permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 } }
I20260812 07:00:56.314265 24886 catalog_manager.cc:5495] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb reported cstate change: term changed from 0 to 1, leader changed from <none> to 77ba4bf54855463689bf0d4a67d50abb (127.26.226.2). New cstate: current_term: 1 leader_uuid: "77ba4bf54855463689bf0d4a67d50abb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77ba4bf54855463689bf0d4a67d50abb" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 } health_report { overall_health: UNKNOWN } } }
I20260812 07:00:56.355808 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.349730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42110 (local address 127.26.226.2:33867)
0812 07:00:56.349990 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:56.349997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:56.350018 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:00:56.350216 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:56.350220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:56.350328 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:56.350509 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:00:56.350953 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.351723 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.353399 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.353853 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.354002 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.354547 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:00:56.355119 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:00:56.355301 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:56.355421 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 07:00:56.355508 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 07:00:56.364686 25105 raft_consensus.cc:1261] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac [term 1 FOLLOWER]: Refusing update from remote peer 77ba4bf54855463689bf0d4a67d50abb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:00:56.364769 25512 raft_consensus.cc:1261] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071 [term 1 FOLLOWER]: Refusing update from remote peer 77ba4bf54855463689bf0d4a67d50abb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:00:56.365832 25627 consensus_queue.cc:1035] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fd1ed5419154459afef529ff6c0a071" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 39005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:00:56.366235 25620 consensus_queue.cc:1035] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [LEADER]: Connected to new peer: Peer: permanent_uuid: "36953873942542b99f3b1a08272e2fac" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 32831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:00:56.379316 25630 mvcc.cc:205] Tried to move back new op lower bound from 7317577958858903552 to 7317577958646779904. Current Snapshot: MvccSnapshot[applied={T|T < 7317577958858903552}]
I20260812 07:00:56.381739 25632 mvcc.cc:205] Tried to move back new op lower bound from 7317577958858903552 to 7317577958646779904. Current Snapshot: MvccSnapshot[applied={T|T < 7317577958858903552}]
I20260812 07:00:56.403704 24885 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:00:56.404089 24885 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:00:56.408836 24885 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf8a78a7aa3d4649a522488ab7a81202: Sending DeleteTablet for 3 replicas of tablet 414ef0dd0ced48e2ad0261af51f4871c
I20260812 07:00:56.410356 25481 tablet_service.cc:1514] Processing DeleteTablet for tablet 414ef0dd0ced48e2ad0261af51f4871c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:00:56 UTC) from {username='slave'} at 127.0.0.1:40012
I20260812 07:00:56.410773 25271 tablet_service.cc:1514] Processing DeleteTablet for tablet 414ef0dd0ced48e2ad0261af51f4871c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:00:56 UTC) from {username='slave'} at 127.0.0.1:42100
I20260812 07:00:56.411674 25078 tablet_service.cc:1514] Processing DeleteTablet for tablet 414ef0dd0ced48e2ad0261af51f4871c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:00:56 UTC) from {username='slave'} at 127.0.0.1:57308
I20260812 07:00:56.411809 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 24987
I20260812 07:00:56.432832 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.413018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57316 (local address 127.26.226.1:32831)
0812 07:00:56.414325 (+ 1307us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:56.414331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:56.423864 (+ 9533us) server_negotiation.cc:407] Connection header received
0812 07:00:56.424419 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:56.424426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:56.424564 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:56.424648 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:56.425388 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.426589 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.428587 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.429270 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.429382 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.430997 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:56.431010 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:56.431031 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:00:56.431831 (+ 800us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:56.431894 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:00:56.431923 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:56.432029 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:56.432318 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 07:00:56.432432 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1170}
I20260812 07:00:56.434340 25165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:56.442075 25641 tablet_replica.cc:330] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: stopping tablet replica
I20260812 07:00:56.442270 25642 tablet_replica.cc:330] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: stopping tablet replica
I20260812 07:00:56.442919 25641 raft_consensus.cc:2229] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:00:56.443257 25642 raft_consensus.cc:2229] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:00:56.443601 25641 raft_consensus.cc:2258] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:00:56.443892 25642 raft_consensus.cc:2258] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:00:56.446014 25641 ts_tablet_manager.cc:1905] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:00:56.446331 25642 ts_tablet_manager.cc:1905] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:00:56.453742 25641 ts_tablet_manager.cc:1918] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:00:56.453989 25641 log.cc:1196] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/414ef0dd0ced48e2ad0261af51f4871c
I20260812 07:00:56.454015 25642 ts_tablet_manager.cc:1918] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:00:56.454327 25642 log.cc:1196] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/414ef0dd0ced48e2ad0261af51f4871c
I20260812 07:00:56.454614 25641 ts_tablet_manager.cc:1939] T 414ef0dd0ced48e2ad0261af51f4871c P 3fd1ed5419154459afef529ff6c0a071: Deleting consensus metadata
I20260812 07:00:56.455013 25642 ts_tablet_manager.cc:1939] T 414ef0dd0ced48e2ad0261af51f4871c P 77ba4bf54855463689bf0d4a67d50abb: Deleting consensus metadata
I20260812 07:00:56.456919 24862 catalog_manager.cc:4841] TS 77ba4bf54855463689bf0d4a67d50abb (127.26.226.2:33867): tablet 414ef0dd0ced48e2ad0261af51f4871c (table test-table [id=4193351e124a4ab580c934bf469b9b91]) successfully deleted
I20260812 07:00:56.456955 24864 catalog_manager.cc:4841] TS 3fd1ed5419154459afef529ff6c0a071 (127.26.226.3:39005): tablet 414ef0dd0ced48e2ad0261af51f4871c (table test-table [id=4193351e124a4ab580c934bf469b9b91]) successfully deleted
I20260812 07:00:56.537689 25643 tablet_replica.cc:330] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac: stopping tablet replica
I20260812 07:00:56.538120 25643 raft_consensus.cc:2229] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:00:56.538538 25643 raft_consensus.cc:2258] T 414ef0dd0ced48e2ad0261af51f4871c P 36953873942542b99f3b1a08272e2fac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:00:56.538949 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 24987
I20260812 07:00:56.564247 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 25184
W20260812 07:00:56.564323 24862 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.226.1:32831 (error 108)
I20260812 07:00:56.566872 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.565853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:32831 (local address 127.0.0.1:57324)
0812 07:00:56.566498 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.566671 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:32831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":406,"threads_started":1}
W20260812 07:00:56.567179 24862 catalog_manager.cc:4584] TS 36953873942542b99f3b1a08272e2fac (127.26.226.1:32831): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 414ef0dd0ced48e2ad0261af51f4871c: Network error: Client connection negotiation failed: client connection to 127.26.226.1:32831: connect: Connection refused (error 111)
I20260812 07:00:56.572495 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.565555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42114 (local address 127.26.226.2:33867)
0812 07:00:56.565725 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:56.565729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:56.565749 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:00:56.565808 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:56.565811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:56.565887 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:56.566027 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:56.566550 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.567695 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.569246 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.569708 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.569854 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.570988 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:56.570998 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:56.571012 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:00:56.571638 (+ 626us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:56.571694 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:00:56.571713 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:56.571863 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:56.572017 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 07:00:56.572135 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:00:56.573271 25378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:56.620957 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.620489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:32831 (local address 127.0.0.1:57334)
0812 07:00:56.620701 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.620805 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:32831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 07:00:56.622109 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.621736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:32831 (local address 127.0.0.1:57350)
0812 07:00:56.621896 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.621975 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:32831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 07:00:56.682617 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.682106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:32831 (local address 127.0.0.1:57362)
0812 07:00:56.682364 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.682455 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:32831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260812 07:00:56.683715 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.683321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:32831 (local address 127.0.0.1:57374)
0812 07:00:56.683480 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.683562 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:32831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 07:00:56.688169 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 25184
I20260812 07:00:56.715641 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 25391
I20260812 07:00:56.723165 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.716563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40022 (local address 127.26.226.3:39005)
0812 07:00:56.716777 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:56.716785 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:56.716879 (+ 94us) server_negotiation.cc:407] Connection header received
0812 07:00:56.717096 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:56.717101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:56.717215 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:56.717337 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:56.717773 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.718488 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.719834 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:56.720461 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:56.720607 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:56.721745 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:56.721754 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:56.721771 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:00:56.722343 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:56.722394 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:00:56.722414 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:56.722556 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:56.722720 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 07:00:56.722834 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 07:00:56.723872 25580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:56.764875 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.764396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:32831 (local address 127.0.0.1:57390)
0812 07:00:56.764612 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.764715 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:32831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 07:00:56.766011 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:56.765570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:32831 (local address 127.0.0.1:57400)
0812 07:00:56.765782 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:56.765863 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:32831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260812 07:00:56.822263 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 25391
I20260812 07:00:56.847064 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 24837
I20260812 07:00:56.848253 24940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:00:56.961876 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 24837
2026-08-12T07:00:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3171 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:00:57 dist-test-slave-t3q3 krb5kdc[25655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:00:57 dist-test-slave-t3q3 krb5kdc[25655](info): set up 2 sockets
Aug 12 07:00:57 dist-test-slave-t3q3 krb5kdc[25655](info): commencing operation
krb5kdc: starting...
W20260812 07:00:58.353063 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:00:58 dist-test-slave-t3q3 krb5kdc[25655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:00:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:00:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:00:58.691329 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33569
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:33569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:00:59.059806 25678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:59.060230 25678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:59.060416 25678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:59.067755 25678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:00:59.067874 25678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:59.067974 25678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:59.068118 25678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:00:59.068181 25678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:00:59.078470 25678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:33569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:59.079964 25678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:59.081390 25678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:59.094264 25689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:59.094347 25686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:59.094280 25684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:59.095784 25678 server_base.cc:1034] running on GCE node
I20260812 07:00:59.096791 25678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:59.098412 25678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:59.099707 25678 hybrid_clock.cc:648] HybridClock initialized: now 1786518059099627 us; error 59 us; skew 500 ppm
I20260812 07:00:59.100185 25678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:59 dist-test-slave-t3q3 krb5kdc[25655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:59.109021 25678 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:00:59.110810 25678 webserver.cc:460] Webserver started at http://127.26.226.62:34701/ using document root <none> and password file <none>
I20260812 07:00:59.111606 25678 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:59.111788 25678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:59.112246 25678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:00:59.115634 25678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "4ac3aecb24d7468fa8f3b9568b84e26c"
format_stamp: "Formatted at 2026-08-12 07:00:59 on dist-test-slave-t3q3"
server_key: "74f40cda1ac4abd53ae391931134e293"
server_key_iv: "f37a366535242dac513f73f0c1b2f214"
server_key_version: "encryptionkey@0"
I20260812 07:00:59.116566 25678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "4ac3aecb24d7468fa8f3b9568b84e26c"
format_stamp: "Formatted at 2026-08-12 07:00:59 on dist-test-slave-t3q3"
server_key: "74f40cda1ac4abd53ae391931134e293"
server_key_iv: "f37a366535242dac513f73f0c1b2f214"
server_key_version: "encryptionkey@0"
I20260812 07:00:59.122359 25678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:00:59.126305 25700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:59.127931 25678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:00:59.128224 25678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "4ac3aecb24d7468fa8f3b9568b84e26c"
format_stamp: "Formatted at 2026-08-12 07:00:59 on dist-test-slave-t3q3"
server_key: "74f40cda1ac4abd53ae391931134e293"
server_key_iv: "f37a366535242dac513f73f0c1b2f214"
server_key_version: "encryptionkey@0"
I20260812 07:00:59.128464 25678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:59.148605 25678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:59.153260 25678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:59.153604 25678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:59.174140 25678 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:33569
I20260812 07:00:59.174170 25781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:33569 every 8 connection(s)
I20260812 07:00:59.175698 25678 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:00:59.177454 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 25678
I20260812 07:00:59.177655 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:00:59.178153 27528 external_mini_cluster.cc:1428] Setting key 5ede26f030ee81ff10c9bbb93b1ec8b9
I20260812 07:00:59.180289 25782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 07:00:59 dist-test-slave-t3q3 krb5kdc[25655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:00:59.194403 25782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c: Bootstrap starting.
I20260812 07:00:59.198778 25782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c: Neither blocks nor log segments found. Creating new log.
I20260812 07:00:59.200250 25782 log.cc:824] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:00:59.204047 25782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c: No bootstrap required, opened a new log
I20260812 07:00:59.205209 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:59.180190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.26.226.62:33569)
0812 07:00:59.181096 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:59.181112 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:59.181163 (+ 51us) server_negotiation.cc:407] Connection header received
0812 07:00:59.183456 (+ 2293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:59.183484 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:59.184004 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:59.184476 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:59.185946 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:59.188094 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:59.189509 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:59.190768 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:59.190999 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:59.195565 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:59.195578 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:59.195597 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:59.195647 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:59.199855 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:59.200819 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:59.200833 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:59.200847 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:59.201142 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:59.201662 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:59.201665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:59.201667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:59.202448 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:59.202754 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:59.203246 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 07:00:59.203701 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":396,"threads_started":1}
I20260812 07:00:59.212772 25782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac3aecb24d7468fa8f3b9568b84e26c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33569 } }
I20260812 07:00:59.213188 25782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:00:59.213274 25782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac3aecb24d7468fa8f3b9568b84e26c, State: Initialized, Role: FOLLOWER
I20260812 07:00:59.213838 25782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac3aecb24d7468fa8f3b9568b84e26c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33569 } }
I20260812 07:00:59.214057 25782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:00:59.214156 25782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:00:59.214280 25782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:00:59.216574 25782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac3aecb24d7468fa8f3b9568b84e26c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33569 } }
I20260812 07:00:59.217041 25782 leader_election.cc:302] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ac3aecb24d7468fa8f3b9568b84e26c; no voters:
I20260812 07:00:59.217577 25782 leader_election.cc:288] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:00:59.217746 25790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:00:59.219041 25790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [term 1 LEADER]: Becoming Leader. State: Replica: 4ac3aecb24d7468fa8f3b9568b84e26c, State: Running, Role: LEADER
I20260812 07:00:59.219724 25790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac3aecb24d7468fa8f3b9568b84e26c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33569 } }
I20260812 07:00:59.219873 25782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:00:59.222711 25793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ac3aecb24d7468fa8f3b9568b84e26c. Latest consensus state: current_term: 1 leader_uuid: "4ac3aecb24d7468fa8f3b9568b84e26c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac3aecb24d7468fa8f3b9568b84e26c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33569 } } }
I20260812 07:00:59.223110 25793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [sys.catalog]: This master's current role is: LEADER
I20260812 07:00:59.222997 25791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ac3aecb24d7468fa8f3b9568b84e26c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac3aecb24d7468fa8f3b9568b84e26c" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33569 } } }
I20260812 07:00:59.223397 25791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c [sys.catalog]: This master's current role is: LEADER
I20260812 07:00:59.225576 25804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:00:59.231137 25804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:00:59.240512 25804 catalog_manager.cc:1347] Generated new cluster ID: db6dcaa6c2d44f8992c219e903d27b70
I20260812 07:00:59.240643 25804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:00:59.260376 25804 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:00:59.262315 25804 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:00:59.277747 25804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c: Generated new TSK 0
I20260812 07:00:59.278728 25804 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 07:00:59.365610 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33569
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:00:59.723115 25821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:00:59.723392 25821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:00:59.723547 25821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:00:59.731139 25821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:00:59.731276 25821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:00:59.731460 25821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 07:00:59.741899 25821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:00:59.743268 25821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:00:59.744692 25821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:00:59.759744 25829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:59.759708 25831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:00:59.760402 25833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:00:59.760562 25821 server_base.cc:1034] running on GCE node
I20260812 07:00:59.761335 25821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:00:59.762444 25821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:00:59.763687 25821 hybrid_clock.cc:648] HybridClock initialized: now 1786518059763619 us; error 61 us; skew 500 ppm
I20260812 07:00:59.764021 25821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:00:59 dist-test-slave-t3q3 krb5kdc[25655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:00:59.767287 25821 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 07:00:59.768891 25821 webserver.cc:460] Webserver started at http://127.26.226.1:37809/ using document root <none> and password file <none>
I20260812 07:00:59.769640 25821 fs_manager.cc:359] Metadata directory not provided
I20260812 07:00:59.769797 25821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:00:59.770125 25821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:00:59.773205 25821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "7072084525f44e188d1f8c81587b33bb"
format_stamp: "Formatted at 2026-08-12 07:00:59 on dist-test-slave-t3q3"
server_key: "ef020f021f35eb87e78cfec7813df40b"
server_key_iv: "41737161d4f6a61389bb4b70ac44f0d7"
server_key_version: "encryptionkey@0"
I20260812 07:00:59.773998 25821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "7072084525f44e188d1f8c81587b33bb"
format_stamp: "Formatted at 2026-08-12 07:00:59 on dist-test-slave-t3q3"
server_key: "ef020f021f35eb87e78cfec7813df40b"
server_key_iv: "41737161d4f6a61389bb4b70ac44f0d7"
server_key_version: "encryptionkey@0"
I20260812 07:00:59.779492 25821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:00:59.783103 25843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:59.784530 25821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:00:59.784718 25821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "7072084525f44e188d1f8c81587b33bb"
format_stamp: "Formatted at 2026-08-12 07:00:59 on dist-test-slave-t3q3"
server_key: "ef020f021f35eb87e78cfec7813df40b"
server_key_iv: "41737161d4f6a61389bb4b70ac44f0d7"
server_key_version: "encryptionkey@0"
I20260812 07:00:59.784925 25821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:00:59.809422 25821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:00:59.813771 25821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:00:59.814184 25821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:00:59.815304 25821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:00:59.816666 25821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:00:59.816746 25821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:59.816834 25821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:00:59.816879 25821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:00:59.852602 25821 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:33803
I20260812 07:00:59.852664 26014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:33803 every 8 connection(s)
I20260812 07:00:59.854012 25821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 07:00:59.861645 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 25821
I20260812 07:00:59.861876 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 07:00:59.862349 27528 external_mini_cluster.cc:1428] Setting key c5282528351fc1adcda6d4edab17de21
Aug 12 07:00:59 dist-test-slave-t3q3 krb5kdc[25655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:00:59.876236 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:59.856775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:41085 (local address 127.26.226.62:33569)
0812 07:00:59.857255 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 07:00:59.857260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:00:59.858254 (+ 994us) server_negotiation.cc:407] Connection header received
0812 07:00:59.860273 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:00:59.860278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:00:59.860372 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:00:59.860496 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:00:59.862204 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:59.863082 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:59.865215 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:00:59.865626 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:00:59.865734 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:59.870162 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:00:59.870167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:00:59.870171 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:00:59.870206 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:00:59.872922 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:59.873739 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:59.873745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:59.873748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:59.873872 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:00:59.874353 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:00:59.874357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:00:59.874359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:00:59.874629 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:00:59.874817 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 07:00:59.875567 (+ 750us) server_negotiation.cc:299] Negotiation successful
0812 07:00:59.875803 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":230,"threads_started":1}
I20260812 07:00:59.877283 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:00:59.857349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33569 (local address 127.26.226.1:41085)
0812 07:00:59.858071 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0812 07:00:59.858116 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:00:59.859927 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:00:59.860760 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:00:59.860768 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:00:59.861315 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:00:59.862000 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:59.862030 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:59.863255 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:59.863263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:59.865047 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:00:59.865065 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:00:59.865883 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:00:59.865893 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:00:59.866244 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:00:59.867022 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:00:59.867059 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:00:59.869909 (+ 2850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:00:59.873094 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:59.873104 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:59.873123 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:59.873582 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:59.873995 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:00:59.874000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:00:59.874003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:00:59.874216 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:00:59.874787 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:00:59.874795 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:00:59.875156 (+ 361us) client_negotiation.cc:769] Sending connection context
0812 07:00:59.875440 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 07:00:59.876020 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":283,"threads_started":1}
I20260812 07:00:59.879029 26015 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33569
I20260812 07:00:59.879387 26015 heartbeater.cc:464] Registering TS with master...
I20260812 07:00:59.880223 26015 heartbeater.cc:511] Master 127.26.226.62:33569 requested a full tablet report, sending...
I20260812 07:00:59.882692 25732 ts_manager.cc:194] Registered new tserver with Master: 7072084525f44e188d1f8c81587b33bb (127.26.226.1:33803)
I20260812 07:00:59.884970 25732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:41085
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 07:00:59.940857 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33569
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:00.300309 26024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:00.300612 26024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:00.300711 26024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:00.308125 26024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:00.308243 26024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:00.308358 26024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 07:01:00.318397 26024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:00.319670 26024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:00.321187 26024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:00.335136 26030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:00.335160 26038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:00.335258 26034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:00.335992 26024 server_base.cc:1034] running on GCE node
I20260812 07:01:00.336684 26024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:00.337610 26024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:00.338843 26024 hybrid_clock.cc:648] HybridClock initialized: now 1786518060338783 us; error 54 us; skew 500 ppm
I20260812 07:01:00.339190 26024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:00 dist-test-slave-t3q3 krb5kdc[25655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518060, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:00.342374 26024 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 07:01:00.343963 26024 webserver.cc:460] Webserver started at http://127.26.226.2:33641/ using document root <none> and password file <none>
I20260812 07:01:00.344695 26024 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:00.344858 26024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:00.345206 26024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:00.348212 26024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "cb22bd1890df46a096f37a08fb841a33"
format_stamp: "Formatted at 2026-08-12 07:01:00 on dist-test-slave-t3q3"
server_key: "996e18f479f59c5c540a4b316f4bcde4"
server_key_iv: "e45a0ab72326f15aca0dfb949010aca5"
server_key_version: "encryptionkey@0"
I20260812 07:01:00.349002 26024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "cb22bd1890df46a096f37a08fb841a33"
format_stamp: "Formatted at 2026-08-12 07:01:00 on dist-test-slave-t3q3"
server_key: "996e18f479f59c5c540a4b316f4bcde4"
server_key_iv: "e45a0ab72326f15aca0dfb949010aca5"
server_key_version: "encryptionkey@0"
I20260812 07:01:00.354390 26024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:01:00.357859 26053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:00.359154 26024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:01:00.359328 26024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "cb22bd1890df46a096f37a08fb841a33"
format_stamp: "Formatted at 2026-08-12 07:01:00 on dist-test-slave-t3q3"
server_key: "996e18f479f59c5c540a4b316f4bcde4"
server_key_iv: "e45a0ab72326f15aca0dfb949010aca5"
server_key_version: "encryptionkey@0"
I20260812 07:01:00.359532 26024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:00.374583 26024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:00.379123 26024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:00.379449 26024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:00.380539 26024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:00.381898 26024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:00.382023 26024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:00.382186 26024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:00.382261 26024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:00.416862 26024 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:33829
I20260812 07:01:00.416913 26239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:33829 every 8 connection(s)
I20260812 07:01:00.418228 26024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 07:01:00.425518 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 26024
I20260812 07:01:00.425714 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 07:01:00.426265 27528 external_mini_cluster.cc:1428] Setting key b34432de53dfb6767e20611b4561e7ce
Aug 12 07:01:00 dist-test-slave-t3q3 krb5kdc[25655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518060, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:00.440358 26243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:00.420973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:39955 (local address 127.26.226.62:33569)
0812 07:01:00.421481 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:00.421486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:00.422408 (+ 922us) server_negotiation.cc:407] Connection header received
0812 07:01:00.424294 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:00.424300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:00.424411 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:00.424547 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:00.426300 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:00.427268 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:00.429355 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:00.429796 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:00.429898 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:00.434266 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:00.434272 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:00.434275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:00.434307 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:00.437130 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:00.437907 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:00.437917 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:00.437920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:00.438053 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:00.438523 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:00.438527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:00.438530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:00.438799 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:00.438964 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:00.439689 (+ 725us) server_negotiation.cc:299] Negotiation successful
0812 07:01:00.439932 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":270,"threads_started":1}
I20260812 07:01:00.441421 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:00.421520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33569 (local address 127.26.226.2:39955)
0812 07:01:00.422240 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:00.422283 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:00.423990 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:00.424808 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:00.424817 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:00.425330 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:00.426108 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:00.426138 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:00.427431 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:00.427435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:00.429185 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:00.429204 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:00.430039 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:00.430050 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:00.430406 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:00.431187 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:00.431224 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:00.434035 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:00.437300 (+ 3265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:00.437309 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:00.437325 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:00.437765 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:00.438173 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:00.438178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:00.438182 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:00.438386 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:00.438936 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:00.438944 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:00.439308 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 07:01:00.439577 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 07:01:00.440156 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":291,"threads_started":1}
I20260812 07:01:00.443046 26240 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33569
I20260812 07:01:00.443421 26240 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:00.444432 26240 heartbeater.cc:511] Master 127.26.226.62:33569 requested a full tablet report, sending...
I20260812 07:01:00.446050 25732 ts_manager.cc:194] Registered new tserver with Master: cb22bd1890df46a096f37a08fb841a33 (127.26.226.2:33829)
I20260812 07:01:00.446868 25732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:39955
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 07:01:00.500298 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33569
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:00.863410 26247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:00.863700 26247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:00.863943 26247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:00.871353 26247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:00.871490 26247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:00.871671 26247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 07:01:00.881803 26247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:00.883170 26247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:00.884579 26247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:01:00.888375 26015 heartbeater.cc:503] Master 127.26.226.62:33569 was elected leader, sending a full tablet report...
W20260812 07:01:00.898830 26254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:00.898890 26255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:00.898913 26261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:00.900287 26247 server_base.cc:1034] running on GCE node
I20260812 07:01:00.901006 26247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:00.902150 26247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:00.903409 26247 hybrid_clock.cc:648] HybridClock initialized: now 1786518060903337 us; error 66 us; skew 500 ppm
I20260812 07:01:00.903771 26247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:00 dist-test-slave-t3q3 krb5kdc[25655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518060, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:00.907142 26247 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 07:01:00.908877 26247 webserver.cc:460] Webserver started at http://127.26.226.3:46209/ using document root <none> and password file <none>
I20260812 07:01:00.909698 26247 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:00.909850 26247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:00.910200 26247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:00.913507 26247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "c83ce68475eb4d5c8e9114197da84024"
format_stamp: "Formatted at 2026-08-12 07:01:00 on dist-test-slave-t3q3"
server_key: "35c2a606f5798fab1321c213061cb72e"
server_key_iv: "6299b90942040bc11f19f34479967e65"
server_key_version: "encryptionkey@0"
I20260812 07:01:00.914352 26247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "c83ce68475eb4d5c8e9114197da84024"
format_stamp: "Formatted at 2026-08-12 07:01:00 on dist-test-slave-t3q3"
server_key: "35c2a606f5798fab1321c213061cb72e"
server_key_iv: "6299b90942040bc11f19f34479967e65"
server_key_version: "encryptionkey@0"
I20260812 07:01:00.920267 26247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260812 07:01:00.923913 26271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:00.925446 26247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:00.925634 26247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "c83ce68475eb4d5c8e9114197da84024"
format_stamp: "Formatted at 2026-08-12 07:01:00 on dist-test-slave-t3q3"
server_key: "35c2a606f5798fab1321c213061cb72e"
server_key_iv: "6299b90942040bc11f19f34479967e65"
server_key_version: "encryptionkey@0"
I20260812 07:01:00.925832 26247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:00.945322 26247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:00.949487 26247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:00.949761 26247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:00.950789 26247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:00.952239 26247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:00.952323 26247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:00.952421 26247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:00.952466 26247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:00.987998 26247 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:36991
I20260812 07:01:00.988042 26445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:36991 every 8 connection(s)
I20260812 07:01:00.989596 26247 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 07:01:00.994493 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 26247
I20260812 07:01:00.994704 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 07:01:00.995178 27528 external_mini_cluster.cc:1428] Setting key 1fe88c2cdf53a581390be8392c369d04
Aug 12 07:01:01 dist-test-slave-t3q3 krb5kdc[25655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518060, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:01.011668 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:00.992357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:47033 (local address 127.26.226.62:33569)
0812 07:01:00.992833 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:00.992837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:00.993863 (+ 1026us) server_negotiation.cc:407] Connection header received
0812 07:01:00.996094 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:00.996099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:00.996258 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:00.996381 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:00.998635 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:00.999455 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.001072 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.001471 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.001570 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.005176 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:01.005181 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:01.005185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:01.005229 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:01.008050 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:01.008963 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:01.008970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:01.008975 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:01.009145 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:01.009711 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:01.009719 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:01.009723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:01.010092 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:01.010254 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.011207 (+ 953us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.011389 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":229,"threads_started":1}
I20260812 07:01:01.012816 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:00.992941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33569 (local address 127.26.226.3:47033)
0812 07:01:00.993681 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:00.993729 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:00.995733 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:00.996764 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:00.996776 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:00.997680 (+ 904us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:00.998439 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:00.998465 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:00.999592 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:00.999595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.000937 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.000951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.001671 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.001676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.001882 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.002568 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:01.002595 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:01.004948 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:01.008244 (+ 3296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:01.008251 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:01.008264 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:01.008808 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:01.009278 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:01.009282 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:01.009285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:01.009557 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:01.010268 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:01.010274 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:01.010728 (+ 454us) client_negotiation.cc:769] Sending connection context
0812 07:01:01.011052 (+ 324us) client_negotiation.cc:240] Negotiation successful
0812 07:01:01.011574 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":275,"threads_started":1}
I20260812 07:01:01.014214 26447 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33569
I20260812 07:01:01.014549 26447 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:01.015328 26447 heartbeater.cc:511] Master 127.26.226.62:33569 requested a full tablet report, sending...
I20260812 07:01:01.016889 25732 ts_manager.cc:194] Registered new tserver with Master: c83ce68475eb4d5c8e9114197da84024 (127.26.226.3:36991)
I20260812 07:01:01.017838 25732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:47033
I20260812 07:01:01.023380 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:01:01.043810 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.028023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.26.226.62:33569)
0812 07:01:01.028245 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:01.028250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:01.028755 (+ 505us) server_negotiation.cc:407] Connection header received
0812 07:01:01.028961 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:01.028966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:01.029063 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:01.029165 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:01.030838 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.031651 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.033376 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.033861 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.033971 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.037048 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:01.037054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:01.037058 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:01.037104 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:01.040684 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:01.041614 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:01.041618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:01.041621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:01.041755 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:01.042371 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:01.042376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:01.042378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:01.042726 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:01.042946 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.043388 (+ 442us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.043551 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260812 07:01:01.048224 25732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:01:01.056396 25732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:01:01.081573 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.071930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:33803 (local address 127.0.0.1:54194)
0812 07:01:01.072668 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:01.072694 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:01.072812 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:01.074120 (+ 1308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:01.074126 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:01.074149 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:01.074668 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.074683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.077033 (+ 2350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.077042 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.079114 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.079127 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.080600 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.080609 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.080778 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.080805 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 07:01:01.080889 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 07:01:01.081120 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":27,"thread_start_us":950,"threads_started":1}
I20260812 07:01:01.081588 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.069879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37956 (local address 127.26.226.2:33829)
0812 07:01:01.070694 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:01.070703 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:01.070746 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:01:01.071000 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:01.071008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:01.071278 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:01.071485 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:01.072762 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.074975 (+ 2213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.078385 (+ 3410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.080219 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.080375 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.080598 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.081004 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.081218 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":470,"threads_started":1}
I20260812 07:01:01.081879 26473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.072633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.26.226.1:33803)
0812 07:01:01.073474 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:01.073484 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:01.073515 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:01:01.073626 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:01.073634 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:01.073888 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:01.074913 (+ 1025us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:01.075128 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.076875 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.079291 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.080338 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.081054 (+ 716us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.081287 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.081356 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.081524 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":440,"threads_started":1}
I20260812 07:01:01.082290 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.069707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33829 (local address 127.0.0.1:37956)
0812 07:01:01.070509 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:01.070560 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:01.070753 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:01.071806 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:01.071818 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:01.071869 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:01.072473 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.072505 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.075144 (+ 2639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.075149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.077855 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.077871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.080473 (+ 2602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.080481 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.080702 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.080759 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 07:01:01.080923 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 07:01:01.081901 (+ 978us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":3,"thread_start_us":339,"threads_started":1}
I20260812 07:01:01.084641 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.071015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:36991 (local address 127.0.0.1:52892)
0812 07:01:01.071854 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:01.071879 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:01.072003 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:01.076156 (+ 4153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:01.076160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:01.076186 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:01.076606 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.076617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.078658 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.078664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.081005 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.081022 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.084046 (+ 3024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.084051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.084237 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.084262 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:01:01.084330 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 07:01:01.084417 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":525,"threads_started":1}
I20260812 07:01:01.085587 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.071445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52892 (local address 127.26.226.3:36991)
0812 07:01:01.075675 (+ 4230us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:01.075681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:01.075714 (+ 33us) server_negotiation.cc:407] Connection header received
0812 07:01:01.075788 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:01.075793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:01.075941 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:01.076167 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:01.076800 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.078500 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.081232 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.083819 (+ 2587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.085001 (+ 1182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.085172 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.085235 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.085350 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4082,"thread_start_us":3909,"threads_started":1}
I20260812 07:01:01.086755 26124 tablet_service.cc:1467] Processing CreateTablet for tablet 0deffec7d7db4009b60a3484bfb4fa8b (DEFAULT_TABLE table=test-table [id=a223c028e8864c2ba30287029a56b81f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:01.086755 25921 tablet_service.cc:1467] Processing CreateTablet for tablet 0deffec7d7db4009b60a3484bfb4fa8b (DEFAULT_TABLE table=test-table [id=a223c028e8864c2ba30287029a56b81f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:01.088426 26124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0deffec7d7db4009b60a3484bfb4fa8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:01.088428 25921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0deffec7d7db4009b60a3484bfb4fa8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:01.089056 26343 tablet_service.cc:1467] Processing CreateTablet for tablet 0deffec7d7db4009b60a3484bfb4fa8b (DEFAULT_TABLE table=test-table [id=a223c028e8864c2ba30287029a56b81f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:01.090310 26343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0deffec7d7db4009b60a3484bfb4fa8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:01.102974 26475 tablet_bootstrap.cc:492] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: Bootstrap starting.
I20260812 07:01:01.106312 26475 tablet_bootstrap.cc:654] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:01.107136 26477 tablet_bootstrap.cc:492] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: Bootstrap starting.
I20260812 07:01:01.107939 26475 log.cc:824] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:01.109596 26478 tablet_bootstrap.cc:492] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: Bootstrap starting.
I20260812 07:01:01.111059 26477 tablet_bootstrap.cc:654] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:01.111675 26475 tablet_bootstrap.cc:492] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: No bootstrap required, opened a new log
I20260812 07:01:01.111989 26475 ts_tablet_manager.cc:1397] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:01:01.112432 26477 log.cc:824] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:01.114526 26478 tablet_bootstrap.cc:654] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:01.115847 26477 tablet_bootstrap.cc:492] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: No bootstrap required, opened a new log
I20260812 07:01:01.115810 26478 log.cc:824] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:01.116158 26477 ts_tablet_manager.cc:1397] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260812 07:01:01.119004 26478 tablet_bootstrap.cc:492] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: No bootstrap required, opened a new log
I20260812 07:01:01.119282 26478 ts_tablet_manager.cc:1397] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260812 07:01:01.121470 26475 raft_consensus.cc:348] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 } } peers { permanent_uuid: "c83ce68475eb4d5c8e9114197da84024" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 36991 } } peers { permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 } }
I20260812 07:01:01.121969 26475 raft_consensus.cc:374] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:01.122087 26475 raft_consensus.cc:729] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c83ce68475eb4d5c8e9114197da84024, State: Initialized, Role: FOLLOWER
I20260812 07:01:01.122810 26475 consensus_queue.cc:260] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 } } peers { permanent_uuid: "c83ce68475eb4d5c8e9114197da84024" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 36991 } } peers { permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 } }
I20260812 07:01:01.123957 26447 heartbeater.cc:503] Master 127.26.226.62:33569 was elected leader, sending a full tablet report...
I20260812 07:01:01.126230 26475 ts_tablet_manager.cc:1428] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260812 07:01:01.126948 26477 raft_consensus.cc:348] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 } } peers { permanent_uuid: "c83ce68475eb4d5c8e9114197da84024" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 36991 } } peers { permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 } }
I20260812 07:01:01.127346 26477 raft_consensus.cc:374] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:01.127435 26477 raft_consensus.cc:729] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7072084525f44e188d1f8c81587b33bb, State: Initialized, Role: FOLLOWER
I20260812 07:01:01.128017 26477 consensus_queue.cc:260] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 } } peers { permanent_uuid: "c83ce68475eb4d5c8e9114197da84024" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 36991 } } peers { permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 } }
I20260812 07:01:01.130111 26477 ts_tablet_manager.cc:1428] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260812 07:01:01.130023 26478 raft_consensus.cc:348] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 } } peers { permanent_uuid: "c83ce68475eb4d5c8e9114197da84024" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 36991 } } peers { permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 } }
I20260812 07:01:01.130508 26478 raft_consensus.cc:374] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:01.130609 26478 raft_consensus.cc:729] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb22bd1890df46a096f37a08fb841a33, State: Initialized, Role: FOLLOWER
I20260812 07:01:01.131238 26478 consensus_queue.cc:260] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 } } peers { permanent_uuid: "c83ce68475eb4d5c8e9114197da84024" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 36991 } } peers { permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 } }
I20260812 07:01:01.132441 26240 heartbeater.cc:503] Master 127.26.226.62:33569 was elected leader, sending a full tablet report...
I20260812 07:01:01.133042 26478 ts_tablet_manager.cc:1428] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
W20260812 07:01:01.170428 26241 tablet.cc:2369] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:01:01.241464 26449 tablet.cc:2369] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:01:01.356767 26016 tablet.cc:2369] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:01:01.379148 26490 raft_consensus.cc:482] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:01:01.379467 26490 raft_consensus.cc:504] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 } } peers { permanent_uuid: "c83ce68475eb4d5c8e9114197da84024" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 36991 } } peers { permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 } }
I20260812 07:01:01.381322 26490 leader_election.cc:288] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb22bd1890df46a096f37a08fb841a33 (127.26.226.2:33829), 7072084525f44e188d1f8c81587b33bb (127.26.226.1:33803)
I20260812 07:01:01.388203 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.381856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:33829 (local address 127.26.226.3:56017)
0812 07:01:01.382117 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:01.382136 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:01.382282 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:01.382733 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:01.382739 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:01.382773 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:01.383048 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.383061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.384382 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.384386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.386164 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.386182 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.387562 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.387566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.387712 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.387737 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:01:01.387815 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:01:01.387911 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":2}
I20260812 07:01:01.388320 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.381971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:56017 (local address 127.26.226.2:33829)
0812 07:01:01.382231 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:01.382239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:01.382264 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:01:01.382460 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:01.382466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:01.382568 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:01.382693 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:01.383224 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.384254 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.386381 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.387398 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.387603 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.387680 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.387877 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.388019 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 07:01:01.389441 26473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.382527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:51989 (local address 127.26.226.1:33803)
0812 07:01:01.382735 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:01.382740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:01.383407 (+ 667us) server_negotiation.cc:407] Connection header received
0812 07:01:01.383494 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:01.383501 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:01.383583 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:01.383688 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:01.384246 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.384989 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.387054 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.388348 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.388551 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.388635 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.389051 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.389186 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:01:01.389514 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.382340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:33803 (local address 127.26.226.3:51989)
0812 07:01:01.383291 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:01.383304 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:01.383399 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:01.383709 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:01.383712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:01.383730 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:01.384021 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.384028 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.385191 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.385197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.386863 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:01.386876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.388759 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.388764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:01.388881 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.388904 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:01:01.389022 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 07:01:01.389174 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"thread_start_us":552,"threads_started":1}
I20260812 07:01:01.389406 26162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0deffec7d7db4009b60a3484bfb4fa8b" candidate_uuid: "c83ce68475eb4d5c8e9114197da84024" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb22bd1890df46a096f37a08fb841a33" is_pre_election: true
I20260812 07:01:01.390069 26162 raft_consensus.cc:2454] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c83ce68475eb4d5c8e9114197da84024 in term 0.
I20260812 07:01:01.390550 25950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0deffec7d7db4009b60a3484bfb4fa8b" candidate_uuid: "c83ce68475eb4d5c8e9114197da84024" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7072084525f44e188d1f8c81587b33bb" is_pre_election: true
I20260812 07:01:01.390830 26274 leader_election.cc:302] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c83ce68475eb4d5c8e9114197da84024, cb22bd1890df46a096f37a08fb841a33; no voters:
I20260812 07:01:01.391054 25950 raft_consensus.cc:2454] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c83ce68475eb4d5c8e9114197da84024 in term 0.
I20260812 07:01:01.391273 26490 raft_consensus.cc:2781] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:01:01.391438 26490 raft_consensus.cc:482] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:01:01.391558 26490 raft_consensus.cc:3037] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:01.393994 26490 raft_consensus.cc:504] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 } } peers { permanent_uuid: "c83ce68475eb4d5c8e9114197da84024" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 36991 } } peers { permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 } }
I20260812 07:01:01.394609 26490 leader_election.cc:288] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [CANDIDATE]: Term 1 election: Requested vote from peers cb22bd1890df46a096f37a08fb841a33 (127.26.226.2:33829), 7072084525f44e188d1f8c81587b33bb (127.26.226.1:33803)
I20260812 07:01:01.395273 26162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0deffec7d7db4009b60a3484bfb4fa8b" candidate_uuid: "c83ce68475eb4d5c8e9114197da84024" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb22bd1890df46a096f37a08fb841a33"
I20260812 07:01:01.395414 25950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0deffec7d7db4009b60a3484bfb4fa8b" candidate_uuid: "c83ce68475eb4d5c8e9114197da84024" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7072084525f44e188d1f8c81587b33bb"
I20260812 07:01:01.395563 26162 raft_consensus.cc:3037] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:01.395694 25950 raft_consensus.cc:3037] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:01.398059 25950 raft_consensus.cc:2454] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c83ce68475eb4d5c8e9114197da84024 in term 1.
I20260812 07:01:01.398267 26162 raft_consensus.cc:2454] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c83ce68475eb4d5c8e9114197da84024 in term 1.
I20260812 07:01:01.398674 26274 leader_election.cc:302] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7072084525f44e188d1f8c81587b33bb, c83ce68475eb4d5c8e9114197da84024; no voters:
I20260812 07:01:01.399024 26490 raft_consensus.cc:2781] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:01.399559 26490 raft_consensus.cc:686] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 1 LEADER]: Becoming Leader. State: Replica: c83ce68475eb4d5c8e9114197da84024, State: Running, Role: LEADER
I20260812 07:01:01.400208 26490 consensus_queue.cc:237] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 } } peers { permanent_uuid: "c83ce68475eb4d5c8e9114197da84024" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 36991 } } peers { permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 } }
I20260812 07:01:01.405313 25732 catalog_manager.cc:5495] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 reported cstate change: term changed from 0 to 1, leader changed from <none> to c83ce68475eb4d5c8e9114197da84024 (127.26.226.3). New cstate: current_term: 1 leader_uuid: "c83ce68475eb4d5c8e9114197da84024" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c83ce68475eb4d5c8e9114197da84024" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 } health_report { overall_health: UNKNOWN } } }
I20260812 07:01:01.452347 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.446560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.26.226.3:36991)
0812 07:01:01.446762 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:01.446767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:01.446795 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:01:01.447029 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:01.447033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:01.447164 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:01.447305 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:01:01.447724 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.448648 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.450104 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.450535 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.450638 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.451166 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:01:01.451699 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:01:01.451839 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.451990 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.452033 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:01:01.460389 25950 raft_consensus.cc:1261] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb [term 1 FOLLOWER]: Refusing update from remote peer c83ce68475eb4d5c8e9114197da84024: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:01:01.460450 26162 raft_consensus.cc:1261] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33 [term 1 FOLLOWER]: Refusing update from remote peer c83ce68475eb4d5c8e9114197da84024: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:01:01.461422 26494 consensus_queue.cc:1035] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7072084525f44e188d1f8c81587b33bb" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 33803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:01:01.461833 26490 consensus_queue.cc:1035] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb22bd1890df46a096f37a08fb841a33" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 33829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:01:01.474543 26498 mvcc.cc:205] Tried to move back new op lower bound from 7317577979732000768 to 7317577979500449792. Current Snapshot: MvccSnapshot[applied={T|T < 7317577979732000768}]
I20260812 07:01:01.493044 25732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:01:01.493353 25732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:01:01.497014 25732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ac3aecb24d7468fa8f3b9568b84e26c: Sending DeleteTablet for 3 replicas of tablet 0deffec7d7db4009b60a3484bfb4fa8b
I20260812 07:01:01.498317 26124 tablet_service.cc:1514] Processing DeleteTablet for tablet 0deffec7d7db4009b60a3484bfb4fa8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:01 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:37956
I20260812 07:01:01.498376 26343 tablet_service.cc:1514] Processing DeleteTablet for tablet 0deffec7d7db4009b60a3484bfb4fa8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:01 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:52892
I20260812 07:01:01.499147 25921 tablet_service.cc:1514] Processing DeleteTablet for tablet 0deffec7d7db4009b60a3484bfb4fa8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:01 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:54194
I20260812 07:01:01.500119 26515 tablet_replica.cc:330] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: stopping tablet replica
I20260812 07:01:01.500743 26515 raft_consensus.cc:2229] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:01:01.501437 26515 raft_consensus.cc:2258] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:01.502394 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 25821
I20260812 07:01:01.503798 26515 ts_tablet_manager.cc:1905] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:01.510609 26514 tablet_replica.cc:330] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: stopping tablet replica
I20260812 07:01:01.510922 26513 tablet_replica.cc:330] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: stopping tablet replica
I20260812 07:01:01.511662 26513 raft_consensus.cc:2229] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:01:01.511870 26514 raft_consensus.cc:2229] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:01:01.512256 26515 ts_tablet_manager.cc:1918] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:01:01.512429 26513 raft_consensus.cc:2258] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:01.512494 26515 log.cc:1196] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/0deffec7d7db4009b60a3484bfb4fa8b
I20260812 07:01:01.513109 26515 ts_tablet_manager.cc:1939] T 0deffec7d7db4009b60a3484bfb4fa8b P 7072084525f44e188d1f8c81587b33bb: Deleting consensus metadata
I20260812 07:01:01.513983 26514 raft_consensus.cc:2258] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:01.515086 26513 ts_tablet_manager.cc:1905] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:01.515328 25703 catalog_manager.cc:4841] TS 7072084525f44e188d1f8c81587b33bb (127.26.226.1:33803): tablet 0deffec7d7db4009b60a3484bfb4fa8b (table test-table [id=a223c028e8864c2ba30287029a56b81f]) successfully deleted
I20260812 07:01:01.516426 26514 ts_tablet_manager.cc:1905] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:01.522472 26513 ts_tablet_manager.cc:1918] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:01:01.522745 26513 log.cc:1196] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/0deffec7d7db4009b60a3484bfb4fa8b
I20260812 07:01:01.523384 26513 ts_tablet_manager.cc:1939] T 0deffec7d7db4009b60a3484bfb4fa8b P cb22bd1890df46a096f37a08fb841a33: Deleting consensus metadata
I20260812 07:01:01.524115 26514 ts_tablet_manager.cc:1918] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:01:01.524358 26514 log.cc:1196] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/0deffec7d7db4009b60a3484bfb4fa8b
I20260812 07:01:01.524991 26514 ts_tablet_manager.cc:1939] T 0deffec7d7db4009b60a3484bfb4fa8b P c83ce68475eb4d5c8e9114197da84024: Deleting consensus metadata
Aug 12 07:01:01 dist-test-slave-t3q3 krb5kdc[25655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 07:01:01.525529 25703 catalog_manager.cc:4841] TS cb22bd1890df46a096f37a08fb841a33 (127.26.226.2:33829): tablet 0deffec7d7db4009b60a3484bfb4fa8b (table test-table [id=a223c028e8864c2ba30287029a56b81f]) successfully deleted
I20260812 07:01:01.526748 25703 catalog_manager.cc:4841] TS c83ce68475eb4d5c8e9114197da84024 (127.26.226.3:36991): tablet 0deffec7d7db4009b60a3484bfb4fa8b (table test-table [id=a223c028e8864c2ba30287029a56b81f]) successfully deleted
I20260812 07:01:01.534416 26473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.503743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54204 (local address 127.26.226.1:33803)
0812 07:01:01.503980 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:01.503990 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:01.514408 (+ 10418us) server_negotiation.cc:407] Connection header received
0812 07:01:01.514736 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:01.514741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:01.514827 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:01.514906 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:01.516865 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.518072 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.520046 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.520757 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.520866 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.527067 (+ 6201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:01.527087 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:01.527106 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:01.527153 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:01.531089 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:01.532014 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:01.532024 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:01.532028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:01.532234 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:01.532624 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:01.532630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:01.532633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:01.533167 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:01.533478 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.533765 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.534005 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 07:01:01.535189 26007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:01.656219 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 25821
I20260812 07:01:01.684828 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 26024
Aug 12 07:01:01 dist-test-slave-t3q3 krb5kdc[25655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 07:01:01.703682 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.685708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.26.226.2:33829)
0812 07:01:01.685915 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:01.685921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:01.685944 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:01:01.686054 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:01.686060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:01.686147 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:01.686287 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:01.687897 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.689080 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.690555 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.691229 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.691377 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.696513 (+ 5136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:01.696538 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:01.696560 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:01.696611 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:01.700522 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:01.701343 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:01.701355 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:01.701359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:01.701534 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:01.701921 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:01.701926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:01.701930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:01.702455 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:01.702781 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.703027 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.703284 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 07:01:01.704617 26233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:01.798480 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 26024
I20260812 07:01:01.835486 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 26247
Aug 12 07:01:01 dist-test-slave-t3q3 krb5kdc[25655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 07:01:01.855530 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:01.836469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52906 (local address 127.26.226.3:36991)
0812 07:01:01.836654 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:01.836659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:01.836744 (+ 85us) server_negotiation.cc:407] Connection header received
0812 07:01:01.836965 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:01.836969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:01.837040 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:01.837157 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:01.839161 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.840142 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.842094 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:01.842608 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:01.842736 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:01.848894 (+ 6158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:01.848914 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:01.848935 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:01.848985 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:01.852371 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:01.853201 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:01.853209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:01.853211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:01.853374 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:01.853818 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:01.853829 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:01.853831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:01.854226 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:01.854605 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:01.854888 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 07:01:01.855136 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:01:01.856428 26440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:01.981926 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 26247
I20260812 07:01:02.008057 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 25678
I20260812 07:01:02.009235 25774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:02.122488 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 25678
2026-08-12T07:01:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5169 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T07:01:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:02Z Disabled control of system clock
I20260812 07:01:02.233526 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34123
--webserver_interface=127.26.226.62
--webserver_port=0
--builtin_ntp_servers=127.26.226.20:38523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:34123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:01:02.603060 26525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:02.603360 26525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:02.603444 26525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:02.610777 26525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:02.610898 26525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:02.610939 26525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:02.611001 26525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:02.611053 26525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:02.620664 26525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:34123
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:34123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:02.622004 26525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:02.623813 26525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:02.636245 26542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:02.636288 26536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:02.636310 26539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:02.637326 26525 server_base.cc:1034] running on GCE node
I20260812 07:01:02.638196 26525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:02.639693 26525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:02.640929 26525 hybrid_clock.cc:648] HybridClock initialized: now 1786518062640867 us; error 55 us; skew 500 ppm
I20260812 07:01:02.641309 26525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:01:02.643663 26525 webserver.cc:460] Webserver started at http://127.26.226.62:32807/ using document root <none> and password file <none>
I20260812 07:01:02.644516 26525 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:02.644656 26525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:02.644996 26525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:02.648176 26525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "0fef069c5b0d49bd9939ee85b2cd9317"
format_stamp: "Formatted at 2026-08-12 07:01:02 on dist-test-slave-t3q3"
server_key: "a486b6056df8fe982968d852512b004b"
server_key_iv: "0e0055895b75aacb6362afbc6fa9fc2a"
server_key_version: "encryptionkey@0"
I20260812 07:01:02.649068 26525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "0fef069c5b0d49bd9939ee85b2cd9317"
format_stamp: "Formatted at 2026-08-12 07:01:02 on dist-test-slave-t3q3"
server_key: "a486b6056df8fe982968d852512b004b"
server_key_iv: "0e0055895b75aacb6362afbc6fa9fc2a"
server_key_version: "encryptionkey@0"
I20260812 07:01:02.654542 26525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:01:02.658200 26550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:02.659665 26525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260812 07:01:02.659868 26525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "0fef069c5b0d49bd9939ee85b2cd9317"
format_stamp: "Formatted at 2026-08-12 07:01:02 on dist-test-slave-t3q3"
server_key: "a486b6056df8fe982968d852512b004b"
server_key_iv: "0e0055895b75aacb6362afbc6fa9fc2a"
server_key_version: "encryptionkey@0"
I20260812 07:01:02.660131 26525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:02.677320 26525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:02.678208 26525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:02.678512 26525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:02.698668 26525 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:34123
I20260812 07:01:02.698685 26632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:34123 every 8 connection(s)
I20260812 07:01:02.700112 26525 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:02.703907 26633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:02.708555 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 26525
I20260812 07:01:02.708762 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:02.709335 27528 external_mini_cluster.cc:1428] Setting key 8eac9c2f47d2d4b20342f2787b012a61
I20260812 07:01:02.715760 26633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317: Bootstrap starting.
I20260812 07:01:02.718900 26633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:02.719995 26633 log.cc:824] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:02.723619 26633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317: No bootstrap required, opened a new log
I20260812 07:01:02.724099 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:02.711472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.26.226.62:34123)
0812 07:01:02.712563 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:02.712574 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:02.712623 (+ 49us) server_negotiation.cc:407] Connection header received
0812 07:01:02.714482 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:02.714507 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:02.714978 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:02.715356 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:02.715969 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:02.717371 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:02.718790 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:02.719559 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:02.719773 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:02.721251 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:02.721259 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:02.721272 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:02.721971 (+ 699us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:02.722095 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:02.722122 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:02.722246 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:02.722486 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 07:01:02.722806 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":401,"threads_started":1}
I20260812 07:01:02.731601 26633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fef069c5b0d49bd9939ee85b2cd9317" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34123 } }
I20260812 07:01:02.731976 26633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:02.732167 26633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fef069c5b0d49bd9939ee85b2cd9317, State: Initialized, Role: FOLLOWER
I20260812 07:01:02.732981 26633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fef069c5b0d49bd9939ee85b2cd9317" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34123 } }
I20260812 07:01:02.733314 26633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:02.733471 26633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:02.733680 26633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:02.736922 26633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fef069c5b0d49bd9939ee85b2cd9317" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34123 } }
I20260812 07:01:02.737574 26633 leader_election.cc:302] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fef069c5b0d49bd9939ee85b2cd9317; no voters:
I20260812 07:01:02.738334 26633 leader_election.cc:288] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:02.738468 26639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:02.739235 26639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [term 1 LEADER]: Becoming Leader. State: Replica: 0fef069c5b0d49bd9939ee85b2cd9317, State: Running, Role: LEADER
I20260812 07:01:02.739775 26639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fef069c5b0d49bd9939ee85b2cd9317" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34123 } }
I20260812 07:01:02.740535 26633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:02.742800 26641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fef069c5b0d49bd9939ee85b2cd9317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fef069c5b0d49bd9939ee85b2cd9317" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34123 } } }
I20260812 07:01:02.743320 26641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:02.743851 26642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fef069c5b0d49bd9939ee85b2cd9317. Latest consensus state: current_term: 1 leader_uuid: "0fef069c5b0d49bd9939ee85b2cd9317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fef069c5b0d49bd9939ee85b2cd9317" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 34123 } } }
I20260812 07:01:02.744189 26642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:02.745105 26654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:02.750780 26654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:02.759781 26654 catalog_manager.cc:1347] Generated new cluster ID: 019f791be338478b89649dfdcd0f942d
I20260812 07:01:02.759914 26654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:02.775209 26654 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:02.777143 26654 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:02.787873 26654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317: Generated new TSK 0
I20260812 07:01:02.788792 26654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:02.801621 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34123
--builtin_ntp_servers=127.26.226.20:38523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:01:03.166383 26672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:03.166707 26672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:03.166853 26672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:03.174135 26672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:03.174243 26672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:03.174435 26672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 07:01:03.183873 26672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:03.185297 26672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:03.187139 26672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:03.200485 26688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:03.200524 26686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:03.200557 26683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:03.201349 26672 server_base.cc:1034] running on GCE node
I20260812 07:01:03.201927 26672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:03.202834 26672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:03.204041 26672 hybrid_clock.cc:648] HybridClock initialized: now 1786518063203992 us; error 44 us; skew 500 ppm
I20260812 07:01:03.204427 26672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:01:03.206880 26672 webserver.cc:460] Webserver started at http://127.26.226.1:33841/ using document root <none> and password file <none>
I20260812 07:01:03.207624 26672 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:03.207752 26672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:03.208146 26672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:03.211102 26672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "b3b7d3b7b16f4cf4a01ad281b932832a"
format_stamp: "Formatted at 2026-08-12 07:01:03 on dist-test-slave-t3q3"
server_key: "02c778bf793f8e6225bccfae97f5b9f6"
server_key_iv: "9de65133ffd2879958faf3fac9fbe3ff"
server_key_version: "encryptionkey@0"
I20260812 07:01:03.211936 26672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "b3b7d3b7b16f4cf4a01ad281b932832a"
format_stamp: "Formatted at 2026-08-12 07:01:03 on dist-test-slave-t3q3"
server_key: "02c778bf793f8e6225bccfae97f5b9f6"
server_key_iv: "9de65133ffd2879958faf3fac9fbe3ff"
server_key_version: "encryptionkey@0"
I20260812 07:01:03.217401 26672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 07:01:03.220911 26698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:03.222435 26672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:01:03.222615 26672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "b3b7d3b7b16f4cf4a01ad281b932832a"
format_stamp: "Formatted at 2026-08-12 07:01:03 on dist-test-slave-t3q3"
server_key: "02c778bf793f8e6225bccfae97f5b9f6"
server_key_iv: "9de65133ffd2879958faf3fac9fbe3ff"
server_key_version: "encryptionkey@0"
I20260812 07:01:03.222908 26672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:03.257606 26672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:03.258496 26672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:03.258781 26672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:03.259763 26672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:03.261394 26672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:03.261485 26672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:03.261569 26672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:03.261624 26672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:03.294556 26672 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:38293
I20260812 07:01:03.294651 26899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:38293 every 8 connection(s)
I20260812 07:01:03.295939 26672 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 07:01:03.298242 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 26672
I20260812 07:01:03.298422 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 07:01:03.298916 27528 external_mini_cluster.cc:1428] Setting key 28ed52955315a4480f96e584bddf93dc
I20260812 07:01:03.302047 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34123
--builtin_ntp_servers=127.26.226.20:38523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:01:03.310143 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:03.298830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:54547 (local address 127.26.226.62:34123)
0812 07:01:03.299378 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:03.299385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:03.300726 (+ 1341us) server_negotiation.cc:407] Connection header received
0812 07:01:03.302748 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:03.302756 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:03.302847 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:03.302965 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:03.304035 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:03.305185 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:03.306766 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:03.307480 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:03.307658 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:03.309130 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:03.309140 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:03.309148 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:03.309174 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:03.309233 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:03.309252 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:03.309346 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:03.309564 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 07:01:03.309685 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":281,"threads_started":1}
I20260812 07:01:03.310997 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:03.299442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34123 (local address 127.26.226.1:54547)
0812 07:01:03.300547 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:03.300598 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:03.302479 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:03.303180 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:03.303188 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:03.303277 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:03.303860 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:03.303877 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:03.305347 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:03.305351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:03.306614 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:03.306627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:03.307739 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:03.307744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:03.307959 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:03.308777 (+ 818us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:01:03.308779 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:03.308835 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:01:03.308921 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:03.309357 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:03.309365 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:03.309372 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:01:03.309516 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 07:01:03.309854 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":325,"threads_started":1}
I20260812 07:01:03.312961 26900 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34123
I20260812 07:01:03.313309 26900 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:03.314100 26900 heartbeater.cc:511] Master 127.26.226.62:34123 requested a full tablet report, sending...
I20260812 07:01:03.316277 26577 ts_manager.cc:194] Registered new tserver with Master: b3b7d3b7b16f4cf4a01ad281b932832a (127.26.226.1:38293)
I20260812 07:01:03.317960 26577 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.1:54547
W20260812 07:01:03.655946 26905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:03.656256 26905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:03.656347 26905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:03.663548 26905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:03.663664 26905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:03.663775 26905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 07:01:03.673338 26905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:03.674688 26905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:03.676590 26905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:03.689782 26917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:03.689829 26911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:03.689851 26912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:03.690642 26905 server_base.cc:1034] running on GCE node
I20260812 07:01:03.691359 26905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:03.692461 26905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:03.693704 26905 hybrid_clock.cc:648] HybridClock initialized: now 1786518063693639 us; error 58 us; skew 500 ppm
I20260812 07:01:03.694046 26905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:01:03.696666 26905 webserver.cc:460] Webserver started at http://127.26.226.2:46099/ using document root <none> and password file <none>
I20260812 07:01:03.697448 26905 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:03.697589 26905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:03.697932 26905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:03.701013 26905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "594a4f65c59745e89bc587d2b4428ff9"
format_stamp: "Formatted at 2026-08-12 07:01:03 on dist-test-slave-t3q3"
server_key: "b76d6af2472f3e00b4cc584297a21bff"
server_key_iv: "74113811ab6ffad4293d49b945bf09d9"
server_key_version: "encryptionkey@0"
I20260812 07:01:03.701846 26905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "594a4f65c59745e89bc587d2b4428ff9"
format_stamp: "Formatted at 2026-08-12 07:01:03 on dist-test-slave-t3q3"
server_key: "b76d6af2472f3e00b4cc584297a21bff"
server_key_iv: "74113811ab6ffad4293d49b945bf09d9"
server_key_version: "encryptionkey@0"
I20260812 07:01:03.707644 26905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:01:03.711582 26925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:03.713177 26905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:03.713380 26905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "594a4f65c59745e89bc587d2b4428ff9"
format_stamp: "Formatted at 2026-08-12 07:01:03 on dist-test-slave-t3q3"
server_key: "b76d6af2472f3e00b4cc584297a21bff"
server_key_iv: "74113811ab6ffad4293d49b945bf09d9"
server_key_version: "encryptionkey@0"
I20260812 07:01:03.713626 26905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:03.741873 26905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:03.742861 26905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:03.743204 26905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:03.744354 26905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:03.746011 26905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:03.746140 26905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:03.746284 26905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:03.746371 26905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:03.782821 26905 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:36031
I20260812 07:01:03.782912 27125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:36031 every 8 connection(s)
I20260812 07:01:03.784266 26905 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 07:01:03.788923 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 26905
I20260812 07:01:03.789130 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 07:01:03.789719 27528 external_mini_cluster.cc:1428] Setting key 9d4740d86d05142a9ee67268bd8831d5
I20260812 07:01:03.793087 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34123
--builtin_ntp_servers=127.26.226.20:38523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:01:03.799114 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:03.786877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:36039 (local address 127.26.226.62:34123)
0812 07:01:03.787108 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:03.787115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:03.788617 (+ 1502us) server_negotiation.cc:407] Connection header received
0812 07:01:03.791107 (+ 2490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:03.791112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:03.791182 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:03.791261 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:03.792633 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:03.793458 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:03.796147 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:03.796558 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:03.796676 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:03.798376 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:03.798381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:03.798385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:03.798404 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:03.798437 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:03.798449 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:03.798516 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:03.798792 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 07:01:03.798877 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 07:01:03.800721 27132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:03.787364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34123 (local address 127.26.226.2:36039)
0812 07:01:03.788417 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:03.788486 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:03.790763 (+ 2277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:03.791537 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:03.791547 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:03.791665 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:03.792447 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:03.792474 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:03.793621 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:03.793630 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:03.795940 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:03.795961 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:03.796806 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:03.796815 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:03.797156 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:03.797980 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:01:03.797986 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:03.798050 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:01:03.798162 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:03.798544 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:03.798556 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:03.798567 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:01:03.798773 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 07:01:03.799222 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":289,"threads_started":1}
I20260812 07:01:03.802237 27129 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34123
I20260812 07:01:03.802582 27129 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:03.803547 27129 heartbeater.cc:511] Master 127.26.226.62:34123 requested a full tablet report, sending...
I20260812 07:01:03.805207 26577 ts_manager.cc:194] Registered new tserver with Master: 594a4f65c59745e89bc587d2b4428ff9 (127.26.226.2:36031)
I20260812 07:01:03.806034 26577 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.2:36039
W20260812 07:01:04.154347 27133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:04.154657 27133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:04.154750 27133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:04.162273 27133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:04.162389 27133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:04.162557 27133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 07:01:04.172031 27133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:04.173480 27133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:04.175314 27133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:04.189529 27138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:04.189574 27139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:04.189522 27142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:04.190781 27133 server_base.cc:1034] running on GCE node
I20260812 07:01:04.191445 27133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:04.192461 27133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:04.193665 27133 hybrid_clock.cc:648] HybridClock initialized: now 1786518064193596 us; error 63 us; skew 500 ppm
I20260812 07:01:04.193953 27133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:01:04.200990 27133 webserver.cc:460] Webserver started at http://127.26.226.3:43147/ using document root <none> and password file <none>
I20260812 07:01:04.201676 27133 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:04.201768 27133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:04.202042 27133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:04.205008 27133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "1600026c074c4346aa84b3d37fb35fa3"
format_stamp: "Formatted at 2026-08-12 07:01:04 on dist-test-slave-t3q3"
server_key: "f152294e27e60114ef3cd20ddda9f17f"
server_key_iv: "70f8dddd3f1d183464d2c1ebe25171e1"
server_key_version: "encryptionkey@0"
I20260812 07:01:04.205772 27133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "1600026c074c4346aa84b3d37fb35fa3"
format_stamp: "Formatted at 2026-08-12 07:01:04 on dist-test-slave-t3q3"
server_key: "f152294e27e60114ef3cd20ddda9f17f"
server_key_iv: "70f8dddd3f1d183464d2c1ebe25171e1"
server_key_version: "encryptionkey@0"
I20260812 07:01:04.210965 27133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:01:04.214417 27158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:04.215716 27133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:01:04.215893 27133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "1600026c074c4346aa84b3d37fb35fa3"
format_stamp: "Formatted at 2026-08-12 07:01:04 on dist-test-slave-t3q3"
server_key: "f152294e27e60114ef3cd20ddda9f17f"
server_key_iv: "70f8dddd3f1d183464d2c1ebe25171e1"
server_key_version: "encryptionkey@0"
I20260812 07:01:04.216149 27133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:04.235452 27133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:04.236423 27133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:04.236755 27133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:04.237776 27133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:04.239450 27133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:04.239590 27133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:04.239719 27133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:04.239806 27133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:04.274771 27133 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:42585
I20260812 07:01:04.274834 27342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:42585 every 8 connection(s)
I20260812 07:01:04.276173 27133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 07:01:04.279493 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 27133
I20260812 07:01:04.279701 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 07:01:04.280224 27528 external_mini_cluster.cc:1428] Setting key db7803640dcc2b3ec516f827f783db55
I20260812 07:01:04.289951 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.278809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:38411 (local address 127.26.226.62:34123)
0812 07:01:04.279030 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:04.279035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:04.280933 (+ 1898us) server_negotiation.cc:407] Connection header received
0812 07:01:04.282956 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:04.282962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:04.283059 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:04.283193 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:04.284395 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.285243 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.286946 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.287407 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.287571 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.289017 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:04.289023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:04.289026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:04.289048 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:04.289086 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:04.289100 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:04.289203 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:04.289450 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 07:01:04.289564 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 07:01:04.291036 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.279327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:34123 (local address 127.26.226.3:38411)
0812 07:01:04.280753 (+ 1426us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:04.280806 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:04.282673 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:04.283434 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:04.283443 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:04.283539 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:04.284211 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.284232 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.285420 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.285424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.286785 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.286799 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.287653 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.287662 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.287923 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.288668 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:01:04.288671 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:04.288712 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:01:04.288799 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:04.289206 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:04.289221 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:04.289231 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 07:01:04.289420 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 07:01:04.289843 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"thread_start_us":493,"threads_started":1}
I20260812 07:01:04.292543 27344 heartbeater.cc:347] Connected to a master server at 127.26.226.62:34123
I20260812 07:01:04.292876 27344 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:04.293617 27344 heartbeater.cc:511] Master 127.26.226.62:34123 requested a full tablet report, sending...
I20260812 07:01:04.295281 26577 ts_manager.cc:194] Registered new tserver with Master: 1600026c074c4346aa84b3d37fb35fa3 (127.26.226.3:42585)
I20260812 07:01:04.296108 26577 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.3:38411
I20260812 07:01:04.307230 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:01:04.317096 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.311399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60518 (local address 127.26.226.62:34123)
0812 07:01:04.311593 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:04.311598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:04.312003 (+ 405us) server_negotiation.cc:407] Connection header received
0812 07:01:04.312190 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:04.312194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:04.312257 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:04.312344 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:04.312776 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.313455 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.314860 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.315284 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.315377 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.316494 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:04.316498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:04.316502 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:04.316518 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:04.316551 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:04.316562 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:04.316724 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:04.316827 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 07:01:04.316873 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:01:04.321255 26900 heartbeater.cc:503] Master 127.26.226.62:34123 was elected leader, sending a full tablet report...
I20260812 07:01:04.321240 26577 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:01:04.328756 26577 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:01:04.368551 27363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.340417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:38293 (local address 127.0.0.1:41646)
0812 07:01:04.354493 (+ 14076us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:04.354543 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:04.354738 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:04.359969 (+ 5231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:04.359977 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:04.360010 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:04.360532 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.360546 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.362611 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.362616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.365946 (+ 3330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.365963 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.367460 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.367467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.367673 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.367729 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 07:01:04.367849 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 07:01:04.368117 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11607,"mutex_wait_us":196,"spinlock_wait_cycles":375040,"thread_start_us":11648,"threads_started":1}
I20260812 07:01:04.368922 27367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.341106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:36031 (local address 127.0.0.1:44722)
0812 07:01:04.359855 (+ 18749us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:04.359872 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:04.360108 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:04.360991 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:04.360995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:04.361021 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:04.361486 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.361506 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.363116 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.363124 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.365944 (+ 2820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.365964 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.367915 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.367921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.368161 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.368195 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 07:01:04.368297 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 07:01:04.368431 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18634,"mutex_wait_us":134,"thread_start_us":18127,"threads_started":1}
I20260812 07:01:04.369273 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.341862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.26.226.2:36031)
0812 07:01:04.358049 (+ 16187us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:04.358063 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:04.359973 (+ 1910us) server_negotiation.cc:407] Connection header received
0812 07:01:04.360313 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:04.360322 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:04.360535 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:04.360694 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:04.361688 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.362968 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.366204 (+ 3236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.367702 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.367857 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.368112 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:04.368393 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 07:01:04.368589 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15970,"thread_start_us":15824,"threads_started":1}
I20260812 07:01:04.369439 27369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.340673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41646 (local address 127.26.226.1:38293)
0812 07:01:04.359177 (+ 18504us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:04.359188 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:04.359240 (+ 52us) server_negotiation.cc:407] Connection header received
0812 07:01:04.359388 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:04.359399 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:04.359714 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:04.360784 (+ 1070us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:04.361037 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.362456 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.366234 (+ 3778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.367258 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.367404 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.367704 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:04.368876 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0812 07:01:04.369056 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18290,"thread_start_us":18123,"threads_started":1}
I20260812 07:01:04.370988 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.340580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:42585 (local address 127.0.0.1:56374)
0812 07:01:04.359586 (+ 19006us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:04.359612 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:04.359865 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:04.360870 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:04.360875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:04.360902 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:04.361557 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.361569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.363175 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.363182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.367026 (+ 3844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.367042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.369716 (+ 2674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.369723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.369868 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.369891 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:01:04.370000 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 07:01:04.370127 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18840,"mutex_wait_us":275,"thread_start_us":19007,"threads_started":1}
I20260812 07:01:04.373234 26787 tablet_service.cc:1467] Processing CreateTablet for tablet f08633070eb24826a694f87280660724 (DEFAULT_TABLE table=test-table [id=a8a3564d734046ebb248daa2bc24b640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:04.373230 27011 tablet_service.cc:1467] Processing CreateTablet for tablet f08633070eb24826a694f87280660724 (DEFAULT_TABLE table=test-table [id=a8a3564d734046ebb248daa2bc24b640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:04.373943 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.341348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56374 (local address 127.26.226.3:42585)
0812 07:01:04.358586 (+ 17238us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:04.358597 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:04.359746 (+ 1149us) server_negotiation.cc:407] Connection header received
0812 07:01:04.360114 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:04.360122 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:04.360352 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:04.360524 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:04.361807 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.363003 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.367254 (+ 4251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.369252 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.369440 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.369729 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:04.370051 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 07:01:04.370219 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17042,"thread_start_us":16879,"threads_started":1}
I20260812 07:01:04.374563 26787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08633070eb24826a694f87280660724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:04.374614 27011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08633070eb24826a694f87280660724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:04.377835 27236 tablet_service.cc:1467] Processing CreateTablet for tablet f08633070eb24826a694f87280660724 (DEFAULT_TABLE table=test-table [id=a8a3564d734046ebb248daa2bc24b640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:04.379402 27236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08633070eb24826a694f87280660724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:04.386876 27378 tablet_bootstrap.cc:492] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: Bootstrap starting.
I20260812 07:01:04.389523 27383 tablet_bootstrap.cc:492] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a: Bootstrap starting.
I20260812 07:01:04.390812 27378 tablet_bootstrap.cc:654] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:04.392216 27378 log.cc:824] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:04.393404 27383 tablet_bootstrap.cc:654] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:04.394896 27383 log.cc:824] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:04.395607 27385 tablet_bootstrap.cc:492] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: Bootstrap starting.
I20260812 07:01:04.400250 27385 tablet_bootstrap.cc:654] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:04.401364 27385 log.cc:824] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:04.402190 27378 tablet_bootstrap.cc:492] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: No bootstrap required, opened a new log
I20260812 07:01:04.402532 27378 ts_tablet_manager.cc:1397] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.004s
I20260812 07:01:04.402837 27383 tablet_bootstrap.cc:492] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a: No bootstrap required, opened a new log
I20260812 07:01:04.403116 27383 ts_tablet_manager.cc:1397] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260812 07:01:04.404587 27385 tablet_bootstrap.cc:492] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: No bootstrap required, opened a new log
I20260812 07:01:04.404843 27385 ts_tablet_manager.cc:1397] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260812 07:01:04.410843 27378 raft_consensus.cc:348] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 } } peers { permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 } } peers { permanent_uuid: "594a4f65c59745e89bc587d2b4428ff9" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36031 } }
I20260812 07:01:04.411257 27378 raft_consensus.cc:374] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:04.411366 27378 raft_consensus.cc:729] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 594a4f65c59745e89bc587d2b4428ff9, State: Initialized, Role: FOLLOWER
I20260812 07:01:04.412011 27378 consensus_queue.cc:260] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 } } peers { permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 } } peers { permanent_uuid: "594a4f65c59745e89bc587d2b4428ff9" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36031 } }
I20260812 07:01:04.413254 27129 heartbeater.cc:503] Master 127.26.226.62:34123 was elected leader, sending a full tablet report...
I20260812 07:01:04.414209 27383 raft_consensus.cc:348] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 } } peers { permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 } } peers { permanent_uuid: "594a4f65c59745e89bc587d2b4428ff9" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36031 } }
I20260812 07:01:04.414678 27383 raft_consensus.cc:374] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:04.414772 27383 raft_consensus.cc:729] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3b7d3b7b16f4cf4a01ad281b932832a, State: Initialized, Role: FOLLOWER
I20260812 07:01:04.415277 27378 ts_tablet_manager.cc:1428] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260812 07:01:04.415493 27383 consensus_queue.cc:260] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 } } peers { permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 } } peers { permanent_uuid: "594a4f65c59745e89bc587d2b4428ff9" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36031 } }
I20260812 07:01:04.415701 27385 raft_consensus.cc:348] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 } } peers { permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 } } peers { permanent_uuid: "594a4f65c59745e89bc587d2b4428ff9" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36031 } }
I20260812 07:01:04.416260 27385 raft_consensus.cc:374] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:04.416369 27385 raft_consensus.cc:729] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1600026c074c4346aa84b3d37fb35fa3, State: Initialized, Role: FOLLOWER
I20260812 07:01:04.417500 27383 ts_tablet_manager.cc:1428] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a: Time spent starting tablet: real 0.014s user 0.014s sys 0.002s
I20260812 07:01:04.418287 27385 consensus_queue.cc:260] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 } } peers { permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 } } peers { permanent_uuid: "594a4f65c59745e89bc587d2b4428ff9" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36031 } }
I20260812 07:01:04.420092 27344 heartbeater.cc:503] Master 127.26.226.62:34123 was elected leader, sending a full tablet report...
I20260812 07:01:04.420749 27385 ts_tablet_manager.cc:1428] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
W20260812 07:01:04.527938 27348 tablet.cc:2369] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:01:04.536496 27131 tablet.cc:2369] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:01:04.548487 26901 tablet.cc:2369] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:01:04.674199 27390 raft_consensus.cc:482] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:01:04.674561 27390 raft_consensus.cc:504] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 } } peers { permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 } } peers { permanent_uuid: "594a4f65c59745e89bc587d2b4428ff9" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36031 } }
I20260812 07:01:04.676452 27390 leader_election.cc:288] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3b7d3b7b16f4cf4a01ad281b932832a (127.26.226.1:38293), 1600026c074c4346aa84b3d37fb35fa3 (127.26.226.3:42585)
I20260812 07:01:04.684685 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.677016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:55947 (local address 127.26.226.3:42585)
0812 07:01:04.677280 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:04.677290 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:04.678265 (+ 975us) server_negotiation.cc:407] Connection header received
0812 07:01:04.678507 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:04.678511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:04.678611 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:04.678701 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:04.679615 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.680692 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.682682 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.683525 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.683711 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.683793 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:04.684263 (+ 470us) server_negotiation.cc:299] Negotiation successful
0812 07:01:04.684410 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:01:04.685385 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.676780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:42585 (local address 127.26.226.2:55947)
0812 07:01:04.678156 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:04.678173 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:04.678358 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:04.678856 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:04.678864 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:04.678903 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:04.679426 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.679442 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.680834 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.680844 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.682481 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.682495 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.683759 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.683768 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.684009 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.684102 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 07:01:04.684220 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 07:01:04.684845 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"mutex_wait_us":266,"thread_start_us":525,"threads_started":1}
I20260812 07:01:04.685993 27369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.677374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:54111 (local address 127.26.226.1:38293)
0812 07:01:04.677629 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:04.677638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:04.678022 (+ 384us) server_negotiation.cc:407] Connection header received
0812 07:01:04.678355 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:04.678364 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:04.678485 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:04.678604 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:04.679311 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.680412 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.683662 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.684841 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.685021 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.685099 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:04.685644 (+ 545us) server_negotiation.cc:299] Negotiation successful
0812 07:01:04.685798 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 07:01:04.686722 27394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.676771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:38293 (local address 127.26.226.2:54111)
0812 07:01:04.677877 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:04.677903 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:04.678158 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:04.678652 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:04.678659 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:04.678690 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:04.679108 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.679119 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.680568 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.680578 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.683398 (+ 2820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:04.683420 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.685249 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.685262 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:04.685424 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.685448 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:01:04.686202 (+ 754us) client_negotiation.cc:240] Negotiation successful
0812 07:01:04.686360 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":57,"thread_start_us":1054,"threads_started":1}
I20260812 07:01:04.686699 27274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08633070eb24826a694f87280660724" candidate_uuid: "594a4f65c59745e89bc587d2b4428ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1600026c074c4346aa84b3d37fb35fa3" is_pre_election: true
I20260812 07:01:04.687444 27274 raft_consensus.cc:2454] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 594a4f65c59745e89bc587d2b4428ff9 in term 0.
I20260812 07:01:04.687949 26827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08633070eb24826a694f87280660724" candidate_uuid: "594a4f65c59745e89bc587d2b4428ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" is_pre_election: true
I20260812 07:01:04.688297 26928 leader_election.cc:302] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1600026c074c4346aa84b3d37fb35fa3, 594a4f65c59745e89bc587d2b4428ff9; no voters:
I20260812 07:01:04.688572 26827 raft_consensus.cc:2454] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 594a4f65c59745e89bc587d2b4428ff9 in term 0.
I20260812 07:01:04.688730 27390 raft_consensus.cc:2781] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:01:04.688890 27390 raft_consensus.cc:482] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:01:04.689013 27390 raft_consensus.cc:3037] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:04.691435 27390 raft_consensus.cc:504] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 } } peers { permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 } } peers { permanent_uuid: "594a4f65c59745e89bc587d2b4428ff9" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36031 } }
I20260812 07:01:04.692167 27390 leader_election.cc:288] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [CANDIDATE]: Term 1 election: Requested vote from peers b3b7d3b7b16f4cf4a01ad281b932832a (127.26.226.1:38293), 1600026c074c4346aa84b3d37fb35fa3 (127.26.226.3:42585)
I20260812 07:01:04.692648 26827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08633070eb24826a694f87280660724" candidate_uuid: "594a4f65c59745e89bc587d2b4428ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a"
I20260812 07:01:04.692886 26827 raft_consensus.cc:3037] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:04.693006 27274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08633070eb24826a694f87280660724" candidate_uuid: "594a4f65c59745e89bc587d2b4428ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1600026c074c4346aa84b3d37fb35fa3"
I20260812 07:01:04.693257 27274 raft_consensus.cc:3037] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:04.696425 26827 raft_consensus.cc:2454] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 594a4f65c59745e89bc587d2b4428ff9 in term 1.
I20260812 07:01:04.696743 27274 raft_consensus.cc:2454] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 594a4f65c59745e89bc587d2b4428ff9 in term 1.
I20260812 07:01:04.696967 26930 leader_election.cc:302] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 594a4f65c59745e89bc587d2b4428ff9, b3b7d3b7b16f4cf4a01ad281b932832a; no voters:
I20260812 07:01:04.697367 27390 raft_consensus.cc:2781] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:04.704306 27390 raft_consensus.cc:686] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 1 LEADER]: Becoming Leader. State: Replica: 594a4f65c59745e89bc587d2b4428ff9, State: Running, Role: LEADER
I20260812 07:01:04.705099 27390 consensus_queue.cc:237] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 } } peers { permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 } } peers { permanent_uuid: "594a4f65c59745e89bc587d2b4428ff9" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36031 } }
I20260812 07:01:04.711519 26574 catalog_manager.cc:5495] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 594a4f65c59745e89bc587d2b4428ff9 (127.26.226.2). New cstate: current_term: 1 leader_uuid: "594a4f65c59745e89bc587d2b4428ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "594a4f65c59745e89bc587d2b4428ff9" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 36031 } health_report { overall_health: HEALTHY } } }
I20260812 07:01:04.742909 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.734045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44738 (local address 127.26.226.2:36031)
0812 07:01:04.734347 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:04.734357 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:04.734381 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:01:04.734506 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:04.734515 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:04.734669 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:04.734864 (+ 195us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:01:04.735692 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.737061 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.739356 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.740246 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.740425 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.741182 (+ 757us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:01:04.742083 (+ 901us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:01:04.742311 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:04.742442 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 07:01:04.742504 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 07:01:04.752615 27274 raft_consensus.cc:1261] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3 [term 1 FOLLOWER]: Refusing update from remote peer 594a4f65c59745e89bc587d2b4428ff9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:01:04.752615 26827 raft_consensus.cc:1261] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a [term 1 FOLLOWER]: Refusing update from remote peer 594a4f65c59745e89bc587d2b4428ff9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:01:04.753804 27395 consensus_queue.cc:1035] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1600026c074c4346aa84b3d37fb35fa3" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 42585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:01:04.754192 27390 consensus_queue.cc:1035] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3b7d3b7b16f4cf4a01ad281b932832a" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:01:04.800562 26574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:01:04.800868 26574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:01:04.804497 26574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fef069c5b0d49bd9939ee85b2cd9317: Sending DeleteTablet for 3 replicas of tablet f08633070eb24826a694f87280660724
I20260812 07:01:04.805927 27011 tablet_service.cc:1514] Processing DeleteTablet for tablet f08633070eb24826a694f87280660724 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:04 UTC) from {username='slave'} at 127.0.0.1:44722
I20260812 07:01:04.806205 27236 tablet_service.cc:1514] Processing DeleteTablet for tablet f08633070eb24826a694f87280660724 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:04 UTC) from {username='slave'} at 127.0.0.1:56374
I20260812 07:01:04.806730 26787 tablet_service.cc:1514] Processing DeleteTablet for tablet f08633070eb24826a694f87280660724 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:04 UTC) from {username='slave'} at 127.0.0.1:41646
I20260812 07:01:04.809183 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 26672
I20260812 07:01:04.810768 27418 tablet_replica.cc:330] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: stopping tablet replica
I20260812 07:01:04.811442 27419 tablet_replica.cc:330] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a: stopping tablet replica
I20260812 07:01:04.812186 27416 tablet_replica.cc:330] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: stopping tablet replica
I20260812 07:01:04.812776 27418 raft_consensus.cc:2229] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:01:04.813386 27418 raft_consensus.cc:2258] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:04.814183 27419 raft_consensus.cc:2229] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:01:04.814703 27416 raft_consensus.cc:2229] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:01:04.815569 27416 raft_consensus.cc:2258] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:04.815886 27418 ts_tablet_manager.cc:1905] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:04.816145 27419 raft_consensus.cc:2258] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:04.819515 27416 ts_tablet_manager.cc:1905] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:04.820762 27369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.810076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41654 (local address 127.26.226.1:38293)
0812 07:01:04.810270 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:04.810276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:04.812691 (+ 2415us) server_negotiation.cc:407] Connection header received
0812 07:01:04.812869 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:04.812876 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:04.813004 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:04.813122 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:04.813617 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.814785 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.816745 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.817140 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.817296 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.818734 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:04.818750 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:04.818768 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:04.819642 (+ 874us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:04.819710 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:04.819736 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:04.819826 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:04.820125 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 07:01:04.820246 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 07:01:04.821924 26891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:04.827169 27418 ts_tablet_manager.cc:1918] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:01:04.827564 27418 log.cc:1196] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/f08633070eb24826a694f87280660724
I20260812 07:01:04.828354 27418 ts_tablet_manager.cc:1939] T f08633070eb24826a694f87280660724 P 1600026c074c4346aa84b3d37fb35fa3: Deleting consensus metadata
I20260812 07:01:04.830122 27416 ts_tablet_manager.cc:1918] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:01:04.830379 27416 log.cc:1196] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/f08633070eb24826a694f87280660724
I20260812 07:01:04.830681 26553 catalog_manager.cc:4841] TS 1600026c074c4346aa84b3d37fb35fa3 (127.26.226.3:42585): tablet f08633070eb24826a694f87280660724 (table test-table [id=a8a3564d734046ebb248daa2bc24b640]) successfully deleted
I20260812 07:01:04.831075 27416 ts_tablet_manager.cc:1939] T f08633070eb24826a694f87280660724 P 594a4f65c59745e89bc587d2b4428ff9: Deleting consensus metadata
I20260812 07:01:04.833482 26552 catalog_manager.cc:4841] TS 594a4f65c59745e89bc587d2b4428ff9 (127.26.226.2:36031): tablet f08633070eb24826a694f87280660724 (table test-table [id=a8a3564d734046ebb248daa2bc24b640]) successfully deleted
I20260812 07:01:04.822059 27419 ts_tablet_manager.cc:1905] T f08633070eb24826a694f87280660724 P b3b7d3b7b16f4cf4a01ad281b932832a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:04.924701 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 26672
W20260812 07:01:04.957717 26555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.226.1:38293 (error 108)
I20260812 07:01:04.957947 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 26905
I20260812 07:01:04.959921 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.959146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:38293 (local address 127.0.0.1:41662)
0812 07:01:04.959602 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:04.959757 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":222,"threads_started":1}
W20260812 07:01:04.960191 26555 catalog_manager.cc:4584] TS b3b7d3b7b16f4cf4a01ad281b932832a (127.26.226.1:38293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f08633070eb24826a694f87280660724: Network error: Client connection negotiation failed: client connection to 127.26.226.1:38293: connect: Connection refused (error 111)
I20260812 07:01:04.966298 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:04.958845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44744 (local address 127.26.226.2:36031)
0812 07:01:04.959062 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:04.959069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:04.959090 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:01:04.959309 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:04.959317 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:04.959442 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:04.959549 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:04.960242 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.961140 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.963036 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:04.963583 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:04.963683 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:04.964858 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:04.964867 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:04.964882 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:04.965519 (+ 637us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:04.965571 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:04.965592 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:04.965681 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:04.965846 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 07:01:04.965937 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:01:04.967042 27119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:05.013896 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:05.013429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:38293 (local address 127.0.0.1:41670)
0812 07:01:05.013643 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:05.013742 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 07:01:05.015023 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:05.014595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:38293 (local address 127.0.0.1:41678)
0812 07:01:05.014809 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:05.014878 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 07:01:05.075910 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:05.075397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:38293 (local address 127.0.0.1:41694)
0812 07:01:05.075622 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:05.075730 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 07:01:05.077057 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:05.076672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:38293 (local address 127.0.0.1:41710)
0812 07:01:05.076822 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:05.076896 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 07:01:05.084474 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 26905
I20260812 07:01:05.113513 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 27133
I20260812 07:01:05.120368 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:05.114355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56376 (local address 127.26.226.3:42585)
0812 07:01:05.114564 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:05.114569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:05.114591 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:01:05.114713 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:05.114717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:05.114805 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:05.114902 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:05.115278 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:05.116000 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:05.117428 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:05.117831 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:05.117930 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:05.118961 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:05.118970 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:05.118986 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:05.119567 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:05.119617 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:05.119637 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:05.119729 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:05.119854 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 07:01:05.119948 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 07:01:05.121037 27336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:05.157756 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:05.157161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:38293 (local address 127.0.0.1:41720)
0812 07:01:05.157472 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:05.157585 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 07:01:05.158974 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:05.158534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:38293 (local address 127.0.0.1:41722)
0812 07:01:05.158754 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:05.158829 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260812 07:01:05.211457 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 27133
I20260812 07:01:05.240317 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 26525
I20260812 07:01:05.241549 26627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:05.353637 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:05.352904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:38293 (local address 127.0.0.1:41730)
0812 07:01:05.353313 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:05.353466 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227}
I20260812 07:01:05.353641 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 26525
2026-08-12T07:01:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3225 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:05 dist-test-slave-t3q3 krb5kdc[27435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:05 dist-test-slave-t3q3 krb5kdc[27435](info): set up 2 sockets
Aug 12 07:01:05 dist-test-slave-t3q3 krb5kdc[27435](info): commencing operation
krb5kdc: starting...
W20260812 07:01:06.765164 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.360s user 0.008s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:06 dist-test-slave-t3q3 krb5kdc[27435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:01:07.105684 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:35375
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:35375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:07.489271 27454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:07.489562 27454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:07.489739 27454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:07.496949 27454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:07.497078 27454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:07.497169 27454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:07.497272 27454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:07.497349 27454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:07.507210 27454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:35375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:35375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:07.508553 27454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:07.509894 27454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:07.521878 27463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:07.521960 27464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:07.521973 27466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:07.523243 27454 server_base.cc:1034] running on GCE node
I20260812 07:01:07.524044 27454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:07.525543 27454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:07.526762 27454 hybrid_clock.cc:648] HybridClock initialized: now 1786518067526716 us; error 36 us; skew 500 ppm
I20260812 07:01:07.527140 27454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:07 dist-test-slave-t3q3 krb5kdc[27435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518067, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:07.530686 27454 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:01:07.532384 27454 webserver.cc:460] Webserver started at http://127.26.226.62:36339/ using document root <none> and password file <none>
I20260812 07:01:07.533198 27454 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:07.533320 27454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:07.533589 27454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:07.536691 27454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "0f2e8f133ea6432880144e8944a06366"
format_stamp: "Formatted at 2026-08-12 07:01:07 on dist-test-slave-t3q3"
server_key: "dad308c66240b8b9eddffb5f3fb0996c"
server_key_iv: "433b907e76bafbbaba26cd113d15b163"
server_key_version: "encryptionkey@0"
I20260812 07:01:07.537436 27454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "0f2e8f133ea6432880144e8944a06366"
format_stamp: "Formatted at 2026-08-12 07:01:07 on dist-test-slave-t3q3"
server_key: "dad308c66240b8b9eddffb5f3fb0996c"
server_key_iv: "433b907e76bafbbaba26cd113d15b163"
server_key_version: "encryptionkey@0"
I20260812 07:01:07.542618 27454 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260812 07:01:07.546030 27481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:07.547327 27454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:01:07.547518 27454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "0f2e8f133ea6432880144e8944a06366"
format_stamp: "Formatted at 2026-08-12 07:01:07 on dist-test-slave-t3q3"
server_key: "dad308c66240b8b9eddffb5f3fb0996c"
server_key_iv: "433b907e76bafbbaba26cd113d15b163"
server_key_version: "encryptionkey@0"
I20260812 07:01:07.547732 27454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:07.560202 27454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:07.564664 27454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:07.564962 27454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:07.584146 27454 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:35375
I20260812 07:01:07.584173 27578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:35375 every 8 connection(s)
I20260812 07:01:07.585595 27454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:07.586366 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 27454
I20260812 07:01:07.586586 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:07.587188 27528 external_mini_cluster.cc:1428] Setting key f0f922ec486a9293c7f5d175159ab346
I20260812 07:01:07.590344 27579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 07:01:07 dist-test-slave-t3q3 krb5kdc[27435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:07.603354 27579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366: Bootstrap starting.
I20260812 07:01:07.607669 27579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:07.608999 27579 log.cc:824] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:07.611982 27579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366: No bootstrap required, opened a new log
I20260812 07:01:07.612361 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:07.589202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39138 (local address 127.26.226.62:35375)
0812 07:01:07.590045 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:07.590060 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:07.590114 (+ 54us) server_negotiation.cc:407] Connection header received
0812 07:01:07.592587 (+ 2473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:07.592616 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:07.593132 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:07.593587 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:07.594936 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:07.597026 (+ 2090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:07.598328 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:07.599352 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:07.599542 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:07.603881 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:07.603893 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:07.603913 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:07.603962 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:07.608027 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:07.608740 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:07.608752 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:07.608763 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:07.609001 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:07.609422 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:07.609430 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:07.609436 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:07.610085 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:07.610408 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:07.610913 (+ 505us) server_negotiation.cc:299] Negotiation successful
0812 07:01:07.611333 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":345,"threads_started":1}
I20260812 07:01:07.619901 27579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2e8f133ea6432880144e8944a06366" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35375 } }
I20260812 07:01:07.620366 27579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:07.620484 27579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2e8f133ea6432880144e8944a06366, State: Initialized, Role: FOLLOWER
I20260812 07:01:07.621052 27579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2e8f133ea6432880144e8944a06366" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35375 } }
I20260812 07:01:07.621295 27579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:07.621438 27579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:07.621621 27579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:07.623822 27579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2e8f133ea6432880144e8944a06366" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35375 } }
I20260812 07:01:07.624369 27579 leader_election.cc:302] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f2e8f133ea6432880144e8944a06366; no voters:
I20260812 07:01:07.625066 27579 leader_election.cc:288] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:07.625154 27586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:07.625896 27586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [term 1 LEADER]: Becoming Leader. State: Replica: 0f2e8f133ea6432880144e8944a06366, State: Running, Role: LEADER
I20260812 07:01:07.626470 27586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2e8f133ea6432880144e8944a06366" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35375 } }
I20260812 07:01:07.627159 27579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:07.629379 27589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f2e8f133ea6432880144e8944a06366" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2e8f133ea6432880144e8944a06366" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35375 } } }
I20260812 07:01:07.629837 27589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:07.630041 27592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f2e8f133ea6432880144e8944a06366. Latest consensus state: current_term: 1 leader_uuid: "0f2e8f133ea6432880144e8944a06366" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2e8f133ea6432880144e8944a06366" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35375 } } }
I20260812 07:01:07.630375 27592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:07.631919 27599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:07.637604 27599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:07.647078 27599 catalog_manager.cc:1347] Generated new cluster ID: 1f14f19e53bd42ff8fd694c3c819c4cb
I20260812 07:01:07.647238 27599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:07.676997 27599 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:07.679117 27599 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:07.693684 27599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366: Generated new TSK 0
I20260812 07:01:07.694665 27599 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.226.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
Entry for principal kudu/127.26.226.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab.
I20260812 07:01:07.782060 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35375
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:08.159508 27621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:08.159817 27621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:08.159914 27621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:08.167284 27621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:08.167413 27621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:08.167531 27621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 07:01:08.177639 27621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.1.keytab
--principal=kudu/127.26.226.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:08.178961 27621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:08.180275 27621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:08.194233 27629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:08.194195 27632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:08.194272 27630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:08.195154 27621 server_base.cc:1034] running on GCE node
I20260812 07:01:08.195820 27621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:08.196887 27621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:08.198093 27621 hybrid_clock.cc:648] HybridClock initialized: now 1786518068198038 us; error 46 us; skew 500 ppm
I20260812 07:01:08.198438 27621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:08 dist-test-slave-t3q3 krb5kdc[27435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518068, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:08.201845 27621 init.cc:377] Logged in from keytab as kudu/127.26.226.1@KRBTEST.COM (short username kudu)
I20260812 07:01:08.203308 27621 webserver.cc:460] Webserver started at http://127.26.226.1:36451/ using document root <none> and password file <none>
I20260812 07:01:08.204035 27621 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:08.204205 27621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:08.204492 27621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:08.207530 27621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "eab78ea97bfa472aac5e3f562f144859"
format_stamp: "Formatted at 2026-08-12 07:01:08 on dist-test-slave-t3q3"
server_key: "2694f968e0d66aa4fe26ae938d0694af"
server_key_iv: "d2d0b8274775750184074c01c9785506"
server_key_version: "encryptionkey@0"
I20260812 07:01:08.208400 27621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "eab78ea97bfa472aac5e3f562f144859"
format_stamp: "Formatted at 2026-08-12 07:01:08 on dist-test-slave-t3q3"
server_key: "2694f968e0d66aa4fe26ae938d0694af"
server_key_iv: "d2d0b8274775750184074c01c9785506"
server_key_version: "encryptionkey@0"
I20260812 07:01:08.213868 27621 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260812 07:01:08.217486 27643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:08.218942 27621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:01:08.219085 27621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "eab78ea97bfa472aac5e3f562f144859"
format_stamp: "Formatted at 2026-08-12 07:01:08 on dist-test-slave-t3q3"
server_key: "2694f968e0d66aa4fe26ae938d0694af"
server_key_iv: "d2d0b8274775750184074c01c9785506"
server_key_version: "encryptionkey@0"
I20260812 07:01:08.219272 27621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:08.254750 27621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:08.259565 27621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:08.259850 27621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:08.260946 27621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:08.262415 27621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:08.262530 27621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:08.262692 27621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:08.262764 27621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:08.298771 27621 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:36807
I20260812 07:01:08.298822 27844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:36807 every 8 connection(s)
I20260812 07:01:08.300222 27621 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 07:01:08.303684 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 27621
I20260812 07:01:08.303937 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 07:01:08.304575 27528 external_mini_cluster.cc:1428] Setting key 0cbed342cafc408ed40c84b9a72cbe85
Aug 12 07:01:08 dist-test-slave-t3q3 krb5kdc[27435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518068, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.1@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:08.321401 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:08.302855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:59461 (local address 127.26.226.62:35375)
0812 07:01:08.303365 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:08.303370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:08.304443 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 07:01:08.306373 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:08.306378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:08.306467 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:08.306585 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:08.308193 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:08.308916 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:08.310501 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:08.310905 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:08.311054 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:08.315109 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:08.315115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:08.315118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:08.315152 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:08.318048 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:08.318945 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:08.318955 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:08.318958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:08.319087 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:08.319535 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:08.319544 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:08.319547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:08.319816 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:08.320003 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:08.320780 (+ 777us) server_negotiation.cc:299] Negotiation successful
0812 07:01:08.321032 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":263,"threads_started":1}
I20260812 07:01:08.322403 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:08.303375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:35375 (local address 127.26.226.1:59461)
0812 07:01:08.304276 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:08.304317 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:08.306088 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:08.306850 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:08.306859 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:08.307405 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:08.307990 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:08.308008 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:08.309098 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:08.309103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:08.310347 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:08.310359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:08.311157 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:08.311167 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:08.311582 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:08.312351 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:08.312379 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:08.314868 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:08.318199 (+ 3331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:08.318209 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:08.318226 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:08.318812 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:08.319203 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:08.319209 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:08.319212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:08.319410 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:08.319958 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:08.319966 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:08.320378 (+ 412us) client_negotiation.cc:769] Sending connection context
0812 07:01:08.320662 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 07:01:08.321208 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":361,"threads_started":1}
I20260812 07:01:08.324270 27845 heartbeater.cc:347] Connected to a master server at 127.26.226.62:35375
I20260812 07:01:08.324617 27845 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:08.325594 27845 heartbeater.cc:511] Master 127.26.226.62:35375 requested a full tablet report, sending...
I20260812 07:01:08.328135 27517 ts_manager.cc:194] Registered new tserver with Master: eab78ea97bfa472aac5e3f562f144859 (127.26.226.1:36807)
I20260812 07:01:08.330370 27517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.1@KRBTEST.COM'} at 127.26.226.1:59461
WARNING: no policy specified for kudu/127.26.226.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
Entry for principal kudu/127.26.226.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab.
I20260812 07:01:08.383106 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35375
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:08.745366 27861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:08.745671 27861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:08.745828 27861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:08.753434 27861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:08.753553 27861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:08.753731 27861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 07:01:08.763746 27861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.2.keytab
--principal=kudu/127.26.226.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:08.765139 27861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:08.766487 27861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:08.780275 27869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:08.780330 27873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:08.780241 27866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:08.781355 27861 server_base.cc:1034] running on GCE node
I20260812 07:01:08.782050 27861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:08.783048 27861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:08.784286 27861 hybrid_clock.cc:648] HybridClock initialized: now 1786518068784229 us; error 75 us; skew 500 ppm
I20260812 07:01:08.784662 27861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:08 dist-test-slave-t3q3 krb5kdc[27435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518068, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:08.788874 27861 init.cc:377] Logged in from keytab as kudu/127.26.226.2@KRBTEST.COM (short username kudu)
I20260812 07:01:08.790419 27861 webserver.cc:460] Webserver started at http://127.26.226.2:42285/ using document root <none> and password file <none>
I20260812 07:01:08.791142 27861 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:08.791267 27861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:08.791538 27861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:08.794631 27861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "5d818c13e70e4442aab6cafc035bee4e"
format_stamp: "Formatted at 2026-08-12 07:01:08 on dist-test-slave-t3q3"
server_key: "98cff562abd5f2a17a795f2166658633"
server_key_iv: "a6719cd8eb09181655d02a4ec3dd0475"
server_key_version: "encryptionkey@0"
I20260812 07:01:08.795401 27861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "5d818c13e70e4442aab6cafc035bee4e"
format_stamp: "Formatted at 2026-08-12 07:01:08 on dist-test-slave-t3q3"
server_key: "98cff562abd5f2a17a795f2166658633"
server_key_iv: "a6719cd8eb09181655d02a4ec3dd0475"
server_key_version: "encryptionkey@0"
I20260812 07:01:08.801012 27861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260812 07:01:08.804890 27881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:08.806269 27861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:01:08.806419 27861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "5d818c13e70e4442aab6cafc035bee4e"
format_stamp: "Formatted at 2026-08-12 07:01:08 on dist-test-slave-t3q3"
server_key: "98cff562abd5f2a17a795f2166658633"
server_key_iv: "a6719cd8eb09181655d02a4ec3dd0475"
server_key_version: "encryptionkey@0"
I20260812 07:01:08.806596 27861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:08.829241 27861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:08.834020 27861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:08.834298 27861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:08.835415 27861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:08.836848 27861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:08.836930 27861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:08.837016 27861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:08.837061 27861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:08.872749 27861 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:42107
I20260812 07:01:08.872805 28077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:42107 every 8 connection(s)
I20260812 07:01:08.874293 27861 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 07:01:08.879758 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 27861
I20260812 07:01:08.879974 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 07:01:08.880462 27528 external_mini_cluster.cc:1428] Setting key b2e5df4881ffd88b5053750b4c4fac19
Aug 12 07:01:08 dist-test-slave-t3q3 krb5kdc[27435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518068, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.2@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:08.897588 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:08.877237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:37283 (local address 127.26.226.62:35375)
0812 07:01:08.877755 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:08.877762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:08.878851 (+ 1089us) server_negotiation.cc:407] Connection header received
0812 07:01:08.881670 (+ 2819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:08.881676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:08.881754 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:08.881879 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:08.883671 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:08.884474 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:08.886040 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:08.886481 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:08.886619 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:08.890401 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:08.890410 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:08.890417 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:08.890466 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:08.894217 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:08.894960 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:08.894968 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:08.894974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:08.895137 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:08.895543 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:08.895551 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:08.895557 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:08.895962 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:08.896149 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:08.896853 (+ 704us) server_negotiation.cc:299] Negotiation successful
0812 07:01:08.897117 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":234,"threads_started":1}
I20260812 07:01:08.898613 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:08.877784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:35375 (local address 127.26.226.2:37283)
0812 07:01:08.878649 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:08.878706 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:08.881375 (+ 2669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:08.882142 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:08.882155 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:08.882905 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:08.883485 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:08.883508 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:08.884636 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:08.884641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:08.885880 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:08.885893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:08.886710 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:08.886718 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:08.886954 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:08.887678 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:08.887702 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:08.890158 (+ 2456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:08.894359 (+ 4201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:08.894366 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:08.894380 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:08.894840 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:08.895252 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:08.895255 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:08.895257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:08.895437 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:08.896132 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:08.896138 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:08.896468 (+ 330us) client_negotiation.cc:769] Sending connection context
0812 07:01:08.896719 (+ 251us) client_negotiation.cc:240] Negotiation successful
0812 07:01:08.897132 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":313,"threads_started":1}
I20260812 07:01:08.900087 28078 heartbeater.cc:347] Connected to a master server at 127.26.226.62:35375
I20260812 07:01:08.900450 28078 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:08.901278 28078 heartbeater.cc:511] Master 127.26.226.62:35375 requested a full tablet report, sending...
I20260812 07:01:08.903187 27517 ts_manager.cc:194] Registered new tserver with Master: 5d818c13e70e4442aab6cafc035bee4e (127.26.226.2:42107)
I20260812 07:01:08.904510 27517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.2@KRBTEST.COM'} at 127.26.226.2:37283
WARNING: no policy specified for kudu/127.26.226.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
Entry for principal kudu/127.26.226.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab.
I20260812 07:01:08.956609 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35375
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:09.318248 28090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:09.318537 28090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:09.318634 28090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:09.326272 28090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:09.326395 28090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:09.326509 28090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 07:01:09.334617 27845 heartbeater.cc:503] Master 127.26.226.62:35375 was elected leader, sending a full tablet report...
I20260812 07:01:09.337762 28090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.3.keytab
--principal=kudu/127.26.226.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:35375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:09.339110 28090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:09.340500 28090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:09.353967 28106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:09.353981 28102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:09.353847 28100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:09.354836 28090 server_base.cc:1034] running on GCE node
I20260812 07:01:09.355486 28090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:09.356525 28090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:09.357762 28090 hybrid_clock.cc:648] HybridClock initialized: now 1786518069357721 us; error 49 us; skew 500 ppm
I20260812 07:01:09.358096 28090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:09 dist-test-slave-t3q3 krb5kdc[27435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518069, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:09.364998 28090 init.cc:377] Logged in from keytab as kudu/127.26.226.3@KRBTEST.COM (short username kudu)
I20260812 07:01:09.366619 28090 webserver.cc:460] Webserver started at http://127.26.226.3:39507/ using document root <none> and password file <none>
I20260812 07:01:09.367393 28090 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:09.367550 28090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:09.367906 28090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:09.370994 28090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "8d5d78f5bf624c698097b6c606667660"
format_stamp: "Formatted at 2026-08-12 07:01:09 on dist-test-slave-t3q3"
server_key: "d4e6362a206fbaf131f6f9fab7270804"
server_key_iv: "79ef8fd75f21f4a35e8215478782a19d"
server_key_version: "encryptionkey@0"
I20260812 07:01:09.371817 28090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "8d5d78f5bf624c698097b6c606667660"
format_stamp: "Formatted at 2026-08-12 07:01:09 on dist-test-slave-t3q3"
server_key: "d4e6362a206fbaf131f6f9fab7270804"
server_key_iv: "79ef8fd75f21f4a35e8215478782a19d"
server_key_version: "encryptionkey@0"
I20260812 07:01:09.377327 28090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 07:01:09.381018 28114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:09.382360 28090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 07:01:09.382550 28090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "8d5d78f5bf624c698097b6c606667660"
format_stamp: "Formatted at 2026-08-12 07:01:09 on dist-test-slave-t3q3"
server_key: "d4e6362a206fbaf131f6f9fab7270804"
server_key_iv: "79ef8fd75f21f4a35e8215478782a19d"
server_key_version: "encryptionkey@0"
I20260812 07:01:09.382771 28090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:09.400589 28090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:09.404807 28090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:09.405113 28090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:09.406138 28090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:09.407564 28090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:09.407680 28090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:09.407827 28090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:09.407898 28090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:09.442492 28090 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:46495
I20260812 07:01:09.442431 28296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:46495 every 8 connection(s)
I20260812 07:01:09.443849 28090 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 07:01:09.451462 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 28090
I20260812 07:01:09.451687 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 07:01:09.452353 27528 external_mini_cluster.cc:1428] Setting key fecc1c000a4590db1bdcd3d09d0d222e
Aug 12 07:01:09 dist-test-slave-t3q3 krb5kdc[27435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518069, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.3@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:09.464690 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.446539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:46659 (local address 127.26.226.62:35375)
0812 07:01:09.447068 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:09.447075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:09.447997 (+ 922us) server_negotiation.cc:407] Connection header received
0812 07:01:09.449822 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:09.449826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:09.449912 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:09.450013 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:09.451592 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.452871 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.454999 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.455564 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.455676 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.459135 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:09.459140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:09.459143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:09.459188 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:09.461896 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:09.462613 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:09.462617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:09.462619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:09.462738 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:09.463132 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:09.463135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:09.463137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:09.463397 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:09.463522 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:09.464247 (+ 725us) server_negotiation.cc:299] Negotiation successful
0812 07:01:09.464420 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":240,"threads_started":1}
I20260812 07:01:09.465757 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.447091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:35375 (local address 127.26.226.3:46659)
0812 07:01:09.447823 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:09.447869 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:09.449570 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:09.450211 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:09.450220 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:09.450782 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:09.451415 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.451433 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.453044 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.453052 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.454870 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.454885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.455765 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.455770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.455964 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.456711 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:09.456739 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:09.458940 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:09.462035 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:09.462043 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:09.462057 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:09.462505 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:09.462839 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:09.462842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:09.462844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:09.463038 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:09.463505 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:09.463511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:09.463851 (+ 340us) client_negotiation.cc:769] Sending connection context
0812 07:01:09.464158 (+ 307us) client_negotiation.cc:240] Negotiation successful
0812 07:01:09.464573 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":274,"threads_started":1}
I20260812 07:01:09.467303 28297 heartbeater.cc:347] Connected to a master server at 127.26.226.62:35375
I20260812 07:01:09.467700 28297 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:09.468555 28297 heartbeater.cc:511] Master 127.26.226.62:35375 requested a full tablet report, sending...
I20260812 07:01:09.469940 27517 ts_manager.cc:194] Registered new tserver with Master: 8d5d78f5bf624c698097b6c606667660 (127.26.226.3:46495)
I20260812 07:01:09.470717 27517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.226.3@KRBTEST.COM'} at 127.26.226.3:46659
I20260812 07:01:09.480600 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:01:09.497902 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.484860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57160 (local address 127.26.226.62:35375)
0812 07:01:09.485039 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:09.485043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:09.485545 (+ 502us) server_negotiation.cc:407] Connection header received
0812 07:01:09.485752 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:09.485757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:09.485878 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:09.485972 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:09.487515 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.488378 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.489677 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.490075 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.490211 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.492715 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:09.492721 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:09.492724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:09.492756 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:09.495150 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:09.495827 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:09.495833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:09.495837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:09.495967 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:09.496379 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:09.496386 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:09.496390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:09.496726 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:09.496981 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:09.497341 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 07:01:09.497578 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:01:09.501641 27517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:01:09.508523 27517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:01:09.532714 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.521119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.26.226.1:36807)
0812 07:01:09.523623 (+ 2504us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:09.523631 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:09.523657 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:01:09.523754 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:09.523761 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:09.523992 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:09.524652 (+ 660us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:09.525193 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.527210 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.529743 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.531305 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.531454 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.531698 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:09.532203 (+ 505us) server_negotiation.cc:299] Negotiation successful
0812 07:01:09.532369 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2288,"thread_start_us":2075,"threads_started":1}
I20260812 07:01:09.532775 28314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.520976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:36807 (local address 127.0.0.1:52856)
0812 07:01:09.522207 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:09.522253 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:09.522412 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:09.524288 (+ 1876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:09.524292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:09.524308 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:09.524988 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.525004 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.527380 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.527386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.529582 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.529595 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.531504 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.531510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.531758 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.531830 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 07:01:09.531972 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 07:01:09.532221 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1000,"mutex_wait_us":89,"thread_start_us":856,"threads_started":1}
I20260812 07:01:09.532866 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.523243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:42107 (local address 127.0.0.1:39664)
0812 07:01:09.523781 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:09.523800 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:09.523913 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:09.525247 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:09.525251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:09.525277 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:09.525629 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.525638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.527342 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.527348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.530019 (+ 2671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.530037 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.531959 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.531967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.532200 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.532240 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 07:01:09.532336 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 07:01:09.532466 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":211,"thread_start_us":770,"threads_started":1}
I20260812 07:01:09.532905 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.523803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39664 (local address 127.26.226.2:42107)
0812 07:01:09.524659 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:09.524667 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:09.524695 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:01:09.524799 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:09.524806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:09.525023 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:09.525263 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:09.525785 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.527172 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.530264 (+ 3092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.531727 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.531863 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.532055 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:09.532391 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 07:01:09.532560 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":496,"threads_started":1}
I20260812 07:01:09.534014 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.521077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:46495 (local address 127.0.0.1:55086)
0812 07:01:09.522986 (+ 1909us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:09.523006 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:09.523139 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:09.524253 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:09.524264 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:09.524297 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:09.524826 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.524841 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.526951 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.526957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.530106 (+ 3149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.530120 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.533123 (+ 3003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.533130 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.533365 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.533400 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:01:09.533521 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 07:01:09.533655 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1792,"mutex_wait_us":194,"thread_start_us":1644,"threads_started":1}
I20260812 07:01:09.534103 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.521626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.26.226.3:46495)
0812 07:01:09.522978 (+ 1352us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:09.522988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:09.523276 (+ 288us) server_negotiation.cc:407] Connection header received
0812 07:01:09.523395 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:09.523404 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:09.523682 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:09.523888 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:09.525029 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.526782 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.530285 (+ 3503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.531727 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.531863 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.532106 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:09.533631 (+ 1525us) server_negotiation.cc:299] Negotiation successful
0812 07:01:09.533788 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1158,"thread_start_us":967,"threads_started":1}
I20260812 07:01:09.537171 27727 tablet_service.cc:1467] Processing CreateTablet for tablet 342e61130ff746a7a512f2f75ded0cdf (DEFAULT_TABLE table=test-table [id=96879fdb12b64ab59263eec90262125f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:09.537182 27968 tablet_service.cc:1467] Processing CreateTablet for tablet 342e61130ff746a7a512f2f75ded0cdf (DEFAULT_TABLE table=test-table [id=96879fdb12b64ab59263eec90262125f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:09.538729 27727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342e61130ff746a7a512f2f75ded0cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:09.538722 27968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342e61130ff746a7a512f2f75ded0cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:09.542162 28198 tablet_service.cc:1467] Processing CreateTablet for tablet 342e61130ff746a7a512f2f75ded0cdf (DEFAULT_TABLE table=test-table [id=96879fdb12b64ab59263eec90262125f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:09.543401 28198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342e61130ff746a7a512f2f75ded0cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:09.553929 28331 tablet_bootstrap.cc:492] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: Bootstrap starting.
I20260812 07:01:09.554704 28330 tablet_bootstrap.cc:492] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: Bootstrap starting.
I20260812 07:01:09.558364 28331 tablet_bootstrap.cc:654] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:09.558626 28330 tablet_bootstrap.cc:654] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:09.559448 28331 log.cc:824] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:09.560128 28330 log.cc:824] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:09.562445 28331 tablet_bootstrap.cc:492] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: No bootstrap required, opened a new log
I20260812 07:01:09.562726 28331 ts_tablet_manager.cc:1397] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260812 07:01:09.560922 28332 tablet_bootstrap.cc:492] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: Bootstrap starting.
I20260812 07:01:09.569244 28330 tablet_bootstrap.cc:492] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: No bootstrap required, opened a new log
I20260812 07:01:09.569518 28330 ts_tablet_manager.cc:1397] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260812 07:01:09.571753 28332 tablet_bootstrap.cc:654] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:09.573160 28332 log.cc:824] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:09.573015 28331 raft_consensus.cc:348] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab78ea97bfa472aac5e3f562f144859" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36807 } } peers { permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 } } peers { permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 } }
I20260812 07:01:09.573484 28331 raft_consensus.cc:374] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:09.573601 28331 raft_consensus.cc:729] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d818c13e70e4442aab6cafc035bee4e, State: Initialized, Role: FOLLOWER
I20260812 07:01:09.574326 28331 consensus_queue.cc:260] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab78ea97bfa472aac5e3f562f144859" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36807 } } peers { permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 } } peers { permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 } }
I20260812 07:01:09.575748 28078 heartbeater.cc:503] Master 127.26.226.62:35375 was elected leader, sending a full tablet report...
I20260812 07:01:09.576579 28332 tablet_bootstrap.cc:492] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: No bootstrap required, opened a new log
I20260812 07:01:09.576799 28331 ts_tablet_manager.cc:1428] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260812 07:01:09.576858 28332 ts_tablet_manager.cc:1397] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.006s
I20260812 07:01:09.578495 28330 raft_consensus.cc:348] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab78ea97bfa472aac5e3f562f144859" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36807 } } peers { permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 } } peers { permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 } }
I20260812 07:01:09.578856 28330 raft_consensus.cc:374] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:09.578964 28330 raft_consensus.cc:729] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab78ea97bfa472aac5e3f562f144859, State: Initialized, Role: FOLLOWER
I20260812 07:01:09.579548 28330 consensus_queue.cc:260] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab78ea97bfa472aac5e3f562f144859" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36807 } } peers { permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 } } peers { permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 } }
I20260812 07:01:09.581843 28330 ts_tablet_manager.cc:1428] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260812 07:01:09.588042 28332 raft_consensus.cc:348] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab78ea97bfa472aac5e3f562f144859" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36807 } } peers { permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 } } peers { permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 } }
I20260812 07:01:09.588572 28332 raft_consensus.cc:374] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:09.588696 28332 raft_consensus.cc:729] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d5d78f5bf624c698097b6c606667660, State: Initialized, Role: FOLLOWER
I20260812 07:01:09.589272 28332 consensus_queue.cc:260] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab78ea97bfa472aac5e3f562f144859" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36807 } } peers { permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 } } peers { permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 } }
I20260812 07:01:09.590317 28297 heartbeater.cc:503] Master 127.26.226.62:35375 was elected leader, sending a full tablet report...
I20260812 07:01:09.591063 28332 ts_tablet_manager.cc:1428] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: Time spent starting tablet: real 0.014s user 0.006s sys 0.006s
W20260812 07:01:09.626649 28080 tablet.cc:2369] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:01:09.695672 28298 tablet.cc:2369] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:01:09.803299 27850 tablet.cc:2369] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:01:09.912705 28342 raft_consensus.cc:482] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:01:09.913019 28342 raft_consensus.cc:504] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab78ea97bfa472aac5e3f562f144859" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36807 } } peers { permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 } } peers { permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 } }
I20260812 07:01:09.914743 28342 leader_election.cc:288] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d5d78f5bf624c698097b6c606667660 (127.26.226.3:46495), 5d818c13e70e4442aab6cafc035bee4e (127.26.226.2:42107)
I20260812 07:01:09.922657 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.915265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:33647 (local address 127.26.226.3:46495)
0812 07:01:09.915527 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:09.915536 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:09.916021 (+ 485us) server_negotiation.cc:407] Connection header received
0812 07:01:09.916280 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:09.916290 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:09.916390 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:09.916490 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:09.917162 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.918144 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.920607 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.921772 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.921945 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.922028 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:09.922271 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 07:01:09.922407 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 07:01:09.922763 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.915178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:46495 (local address 127.26.226.1:33647)
0812 07:01:09.915859 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:09.915876 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:09.916093 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:09.916548 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:09.916553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:09.916580 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:09.917000 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.917013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.918277 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.918281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.920429 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.920445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.921962 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.921966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.922093 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.922118 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:01:09.922216 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 07:01:09.922351 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":382,"threads_started":1}
I20260812 07:01:09.923158 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.915982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:42107 (local address 127.26.226.1:56105)
0812 07:01:09.916704 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:09.916726 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:09.916860 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:09.917322 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:09.917325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:09.917350 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:09.917658 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.917666 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.918818 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.918824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.920803 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:09.920813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.922430 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.922435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:09.922611 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.922640 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:01:09.922761 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 07:01:09.922889 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":357,"threads_started":1}
I20260812 07:01:09.923228 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.916211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:56105 (local address 127.26.226.2:42107)
0812 07:01:09.916441 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:09.916448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:09.916829 (+ 381us) server_negotiation.cc:407] Connection header received
0812 07:01:09.917100 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:09.917107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:09.917188 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:09.917301 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:09.917833 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.918676 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.920954 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.922180 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.922356 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.922421 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:09.922850 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 07:01:09.922975 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 07:01:09.924185 28232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342e61130ff746a7a512f2f75ded0cdf" candidate_uuid: "eab78ea97bfa472aac5e3f562f144859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d5d78f5bf624c698097b6c606667660" is_pre_election: true
I20260812 07:01:09.924412 28003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342e61130ff746a7a512f2f75ded0cdf" candidate_uuid: "eab78ea97bfa472aac5e3f562f144859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d818c13e70e4442aab6cafc035bee4e" is_pre_election: true
I20260812 07:01:09.924700 28232 raft_consensus.cc:2454] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eab78ea97bfa472aac5e3f562f144859 in term 0.
I20260812 07:01:09.924963 28003 raft_consensus.cc:2454] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eab78ea97bfa472aac5e3f562f144859 in term 0.
I20260812 07:01:09.925391 27649 leader_election.cc:302] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d5d78f5bf624c698097b6c606667660, eab78ea97bfa472aac5e3f562f144859; no voters:
I20260812 07:01:09.925781 28342 raft_consensus.cc:2781] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:01:09.925935 28342 raft_consensus.cc:482] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:01:09.926059 28342 raft_consensus.cc:3037] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:09.928361 28342 raft_consensus.cc:504] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab78ea97bfa472aac5e3f562f144859" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36807 } } peers { permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 } } peers { permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 } }
I20260812 07:01:09.928985 28342 leader_election.cc:288] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [CANDIDATE]: Term 1 election: Requested vote from peers 8d5d78f5bf624c698097b6c606667660 (127.26.226.3:46495), 5d818c13e70e4442aab6cafc035bee4e (127.26.226.2:42107)
I20260812 07:01:09.929373 28232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342e61130ff746a7a512f2f75ded0cdf" candidate_uuid: "eab78ea97bfa472aac5e3f562f144859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d5d78f5bf624c698097b6c606667660"
I20260812 07:01:09.929450 28003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342e61130ff746a7a512f2f75ded0cdf" candidate_uuid: "eab78ea97bfa472aac5e3f562f144859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d818c13e70e4442aab6cafc035bee4e"
I20260812 07:01:09.929569 28232 raft_consensus.cc:3037] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:09.929665 28003 raft_consensus.cc:3037] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:09.931774 28232 raft_consensus.cc:2454] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eab78ea97bfa472aac5e3f562f144859 in term 1.
I20260812 07:01:09.931854 28003 raft_consensus.cc:2454] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eab78ea97bfa472aac5e3f562f144859 in term 1.
I20260812 07:01:09.932165 27649 leader_election.cc:302] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d5d78f5bf624c698097b6c606667660, eab78ea97bfa472aac5e3f562f144859; no voters:
I20260812 07:01:09.932451 28342 raft_consensus.cc:2781] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:09.932888 28342 raft_consensus.cc:686] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 1 LEADER]: Becoming Leader. State: Replica: eab78ea97bfa472aac5e3f562f144859, State: Running, Role: LEADER
I20260812 07:01:09.933373 28342 consensus_queue.cc:237] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab78ea97bfa472aac5e3f562f144859" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36807 } } peers { permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 } } peers { permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 } }
I20260812 07:01:09.938233 27516 catalog_manager.cc:5495] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 reported cstate change: term changed from 0 to 1, leader changed from <none> to eab78ea97bfa472aac5e3f562f144859 (127.26.226.1). New cstate: current_term: 1 leader_uuid: "eab78ea97bfa472aac5e3f562f144859" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab78ea97bfa472aac5e3f562f144859" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 36807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 } health_report { overall_health: UNKNOWN } } }
I20260812 07:01:09.995308 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:09.989881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52866 (local address 127.26.226.1:36807)
0812 07:01:09.990120 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:09.990128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:09.990153 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:01:09.990270 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:09.990275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:09.990354 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:09.990514 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:01:09.990898 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.991680 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.993118 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:09.993521 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:09.993662 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:09.994165 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:01:09.994678 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:01:09.994843 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:09.994953 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 07:01:09.995038 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 07:01:10.003665 28003 raft_consensus.cc:1261] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e [term 1 FOLLOWER]: Refusing update from remote peer eab78ea97bfa472aac5e3f562f144859: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:01:10.003695 28232 raft_consensus.cc:1261] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660 [term 1 FOLLOWER]: Refusing update from remote peer eab78ea97bfa472aac5e3f562f144859: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:01:10.004794 28350 consensus_queue.cc:1035] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d818c13e70e4442aab6cafc035bee4e" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 42107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:01:10.005187 28342 consensus_queue.cc:1035] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d5d78f5bf624c698097b6c606667660" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:01:10.016472 28359 mvcc.cc:205] Tried to move back new op lower bound from 7317578014724485120 to 7317578014450855936. Current Snapshot: MvccSnapshot[applied={T|T < 7317578014724485120}]
I20260812 07:01:10.021333 28361 mvcc.cc:205] Tried to move back new op lower bound from 7317578014724485120 to 7317578014450855936. Current Snapshot: MvccSnapshot[applied={T|T < 7317578014724485120}]
I20260812 07:01:10.031090 27516 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:01:10.031389 27516 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:01:10.035037 27516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f2e8f133ea6432880144e8944a06366: Sending DeleteTablet for 3 replicas of tablet 342e61130ff746a7a512f2f75ded0cdf
I20260812 07:01:10.036149 27727 tablet_service.cc:1514] Processing DeleteTablet for tablet 342e61130ff746a7a512f2f75ded0cdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:10 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:52856
I20260812 07:01:10.036418 27968 tablet_service.cc:1514] Processing DeleteTablet for tablet 342e61130ff746a7a512f2f75ded0cdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:10 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:39664
I20260812 07:01:10.037343 28198 tablet_service.cc:1514] Processing DeleteTablet for tablet 342e61130ff746a7a512f2f75ded0cdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:10 UTC) from {username='kudu', principal='kudu/127.26.226.62@KRBTEST.COM'} at 127.0.0.1:55086
I20260812 07:01:10.041285 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 27621
I20260812 07:01:10.046255 28381 tablet_replica.cc:330] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: stopping tablet replica
I20260812 07:01:10.046836 28381 raft_consensus.cc:2229] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:01:10.047518 28381 raft_consensus.cc:2258] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:10.049382 28380 tablet_replica.cc:330] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: stopping tablet replica
I20260812 07:01:10.049893 28380 raft_consensus.cc:2229] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:01:10.050494 28380 raft_consensus.cc:2258] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:10.053113 28380 ts_tablet_manager.cc:1905] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:10.055024 28381 ts_tablet_manager.cc:1905] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:10.060593 28380 ts_tablet_manager.cc:1918] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:01:10.060883 28380 log.cc:1196] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/342e61130ff746a7a512f2f75ded0cdf
I20260812 07:01:10.061581 28380 ts_tablet_manager.cc:1939] T 342e61130ff746a7a512f2f75ded0cdf P 5d818c13e70e4442aab6cafc035bee4e: Deleting consensus metadata
I20260812 07:01:10.063742 28374 tablet_replica.cc:330] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: stopping tablet replica
I20260812 07:01:10.063977 27491 catalog_manager.cc:4841] TS 5d818c13e70e4442aab6cafc035bee4e (127.26.226.2:42107): tablet 342e61130ff746a7a512f2f75ded0cdf (table test-table [id=96879fdb12b64ab59263eec90262125f]) successfully deleted
I20260812 07:01:10.064311 28374 raft_consensus.cc:2229] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:01:10.065039 28374 raft_consensus.cc:2258] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:10.066766 28381 ts_tablet_manager.cc:1918] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:01:10.067027 28381 log.cc:1196] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/342e61130ff746a7a512f2f75ded0cdf
I20260812 07:01:10.067569 28374 ts_tablet_manager.cc:1905] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 07:01:10 dist-test-slave-t3q3 krb5kdc[27435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.1@KRBTEST.COM
I20260812 07:01:10.067646 28381 ts_tablet_manager.cc:1939] T 342e61130ff746a7a512f2f75ded0cdf P 8d5d78f5bf624c698097b6c606667660: Deleting consensus metadata
I20260812 07:01:10.070080 27491 catalog_manager.cc:4841] TS 8d5d78f5bf624c698097b6c606667660 (127.26.226.3:46495): tablet 342e61130ff746a7a512f2f75ded0cdf (table test-table [id=96879fdb12b64ab59263eec90262125f]) successfully deleted
I20260812 07:01:10.075431 28374 ts_tablet_manager.cc:1918] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:01:10.075629 28374 log.cc:1196] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517734245872-27528-0/minicluster-data/ts-0/wal/wals/342e61130ff746a7a512f2f75ded0cdf
I20260812 07:01:10.076195 28374 ts_tablet_manager.cc:1939] T 342e61130ff746a7a512f2f75ded0cdf P eab78ea97bfa472aac5e3f562f144859: Deleting consensus metadata
I20260812 07:01:10.078102 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:10.042240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52878 (local address 127.26.226.1:36807)
0812 07:01:10.042418 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:10.042424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:10.054167 (+ 11743us) server_negotiation.cc:407] Connection header received
0812 07:01:10.055259 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:10.055265 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:10.055369 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:10.055449 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:10.057341 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:10.058649 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:10.060930 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:10.061969 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:10.062106 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:10.069035 (+ 6929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:10.069050 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:10.069068 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:10.069126 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:10.073785 (+ 4659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:10.074780 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:10.074790 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:10.074794 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:10.075025 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:10.075606 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:10.075618 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:10.075621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:10.076236 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:10.076558 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:10.077231 (+ 673us) server_negotiation.cc:299] Negotiation successful
0812 07:01:10.077496 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 07:01:10.078287 27487 catalog_manager.cc:4841] TS eab78ea97bfa472aac5e3f562f144859 (127.26.226.1:36807): tablet 342e61130ff746a7a512f2f75ded0cdf (table test-table [id=96879fdb12b64ab59263eec90262125f]) successfully deleted
I20260812 07:01:10.078919 27838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:10.180392 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 27621
I20260812 07:01:10.208160 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 27861
Aug 12 07:01:10 dist-test-slave-t3q3 krb5kdc[27435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.2@KRBTEST.COM
I20260812 07:01:10.223511 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:10.208926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39672 (local address 127.26.226.2:42107)
0812 07:01:10.209098 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:10.209103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:10.209123 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:01:10.209258 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:10.209262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:10.209337 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:10.209425 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:10.210783 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:10.211618 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:10.213077 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:10.213481 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:10.213581 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:10.217986 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:10.218001 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:10.218019 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:10.218056 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:10.221002 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:10.221667 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:10.221674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:10.221677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:10.221804 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:10.222191 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:10.222194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:10.222196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:10.222573 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:10.222802 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:10.223068 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 07:01:10.223251 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:01:10.224287 28068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:10.315466 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 27861
I20260812 07:01:10.343420 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 28090
Aug 12 07:01:10 dist-test-slave-t3q3 krb5kdc[27435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.3@KRBTEST.COM
I20260812 07:01:10.359139 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:10.344241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55100 (local address 127.26.226.3:46495)
0812 07:01:10.344432 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:10.344441 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:10.344464 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:01:10.344690 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:10.344695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:10.344763 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:10.344871 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:10.346293 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:10.347030 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:10.348491 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:10.348886 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:10.349023 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:10.353607 (+ 4584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:10.353624 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:10.353641 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:10.353684 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:10.356349 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:10.357026 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:10.357033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:10.357036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:10.357173 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:10.357571 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:10.357575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:10.357577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:10.357948 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:10.358247 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:10.358568 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 07:01:10.358801 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:01:10.359817 28289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:10.449406 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 28090
I20260812 07:01:10.477133 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 27454
I20260812 07:01:10.478302 27573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:10.589574 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 27454
2026-08-12T07:01:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5237 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T07:01:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:10Z Disabled control of system clock
I20260812 07:01:10.693195 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33579
--webserver_interface=127.26.226.62
--webserver_port=0
--builtin_ntp_servers=127.26.226.20:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:01:11.063424 28399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:11.063716 28399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:11.063864 28399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:11.071099 28399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:11.071221 28399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:11.071321 28399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:11.071420 28399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:11.071484 28399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:11.080988 28399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:33579
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:33579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:11.082331 28399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:11.084223 28399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:11.096045 28408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:11.096089 28405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:11.095994 28406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:11.097307 28399 server_base.cc:1034] running on GCE node
I20260812 07:01:11.098027 28399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:11.099412 28399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:11.100652 28399 hybrid_clock.cc:648] HybridClock initialized: now 1786518071100613 us; error 49 us; skew 500 ppm
I20260812 07:01:11.101024 28399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:01:11.103372 28399 webserver.cc:460] Webserver started at http://127.26.226.62:34727/ using document root <none> and password file <none>
I20260812 07:01:11.104137 28399 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:11.104274 28399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:11.104605 28399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:11.107550 28399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "7c14f1be78344bab80868fd15907e565"
format_stamp: "Formatted at 2026-08-12 07:01:11 on dist-test-slave-t3q3"
server_key: "aae05b2086ba940ced5952092b1cb9cb"
server_key_iv: "d45f60c48f13f693375f25c1987bf17b"
server_key_version: "encryptionkey@0"
I20260812 07:01:11.108469 28399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "7c14f1be78344bab80868fd15907e565"
format_stamp: "Formatted at 2026-08-12 07:01:11 on dist-test-slave-t3q3"
server_key: "aae05b2086ba940ced5952092b1cb9cb"
server_key_iv: "d45f60c48f13f693375f25c1987bf17b"
server_key_version: "encryptionkey@0"
I20260812 07:01:11.113735 28399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:01:11.117281 28415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:11.118994 28399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:01:11.119199 28399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "7c14f1be78344bab80868fd15907e565"
format_stamp: "Formatted at 2026-08-12 07:01:11 on dist-test-slave-t3q3"
server_key: "aae05b2086ba940ced5952092b1cb9cb"
server_key_iv: "d45f60c48f13f693375f25c1987bf17b"
server_key_version: "encryptionkey@0"
I20260812 07:01:11.119422 28399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:11.138864 28399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:11.139755 28399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:11.140108 28399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:11.159785 28399 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:33579
I20260812 07:01:11.159826 28501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:33579 every 8 connection(s)
I20260812 07:01:11.161232 28399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:11.165050 28503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:11.167760 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 28399
I20260812 07:01:11.167943 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:11.168550 27528 external_mini_cluster.cc:1428] Setting key 80ca710aac90be26c7737823013693e1
I20260812 07:01:11.176992 28503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565: Bootstrap starting.
I20260812 07:01:11.180099 28503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:11.181306 28503 log.cc:824] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:11.182473 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:11.170755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.26.226.62:33579)
0812 07:01:11.171765 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:11.171775 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:11.171816 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:01:11.173561 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:11.173586 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:11.174048 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:11.174439 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:11.175033 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:11.176392 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:11.177764 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:11.178455 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:11.178645 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:11.179824 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:11.179831 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:11.179845 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:11.180480 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:11.180585 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:11.180607 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:11.180732 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:11.180956 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 07:01:11.181266 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":383,"threads_started":1}
I20260812 07:01:11.184541 28503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565: No bootstrap required, opened a new log
I20260812 07:01:11.191978 28503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c14f1be78344bab80868fd15907e565" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33579 } }
I20260812 07:01:11.192420 28503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:11.192503 28503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c14f1be78344bab80868fd15907e565, State: Initialized, Role: FOLLOWER
I20260812 07:01:11.193109 28503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c14f1be78344bab80868fd15907e565" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33579 } }
I20260812 07:01:11.193367 28503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:11.193521 28503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:11.193703 28503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:11.196130 28503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c14f1be78344bab80868fd15907e565" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33579 } }
I20260812 07:01:11.196638 28503 leader_election.cc:302] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c14f1be78344bab80868fd15907e565; no voters:
I20260812 07:01:11.197245 28503 leader_election.cc:288] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:11.197422 28510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:11.198247 28510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [term 1 LEADER]: Becoming Leader. State: Replica: 7c14f1be78344bab80868fd15907e565, State: Running, Role: LEADER
I20260812 07:01:11.198791 28510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c14f1be78344bab80868fd15907e565" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33579 } }
I20260812 07:01:11.199352 28503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:11.201663 28512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c14f1be78344bab80868fd15907e565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c14f1be78344bab80868fd15907e565" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33579 } } }
I20260812 07:01:11.202123 28512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:11.204216 28525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:11.204137 28515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c14f1be78344bab80868fd15907e565. Latest consensus state: current_term: 1 leader_uuid: "7c14f1be78344bab80868fd15907e565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c14f1be78344bab80868fd15907e565" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 33579 } } }
I20260812 07:01:11.204468 28515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:11.209556 28525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:11.210988 28541 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:01:11.219276 28525 catalog_manager.cc:1347] Generated new cluster ID: 634fbf41979940aab92a64c12be15b5e
I20260812 07:01:11.219399 28525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:11.233224 28525 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:11.235090 28525 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:11.242759 28525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565: Generated new TSK 0
I20260812 07:01:11.243829 28525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:11.255237 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.1:0
--local_ip_for_outbound_sockets=127.26.226.1
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33579
--builtin_ntp_servers=127.26.226.20:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:01:11.612509 28544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:11.612785 28544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:11.612946 28544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:11.620400 28544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:11.620517 28544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:11.620688 28544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.1
I20260812 07:01:11.629993 28544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.226.1
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.1
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:11.631232 28544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:11.633168 28544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:11.646127 28553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:11.646087 28555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:11.646377 28558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:11.647030 28544 server_base.cc:1034] running on GCE node
I20260812 07:01:11.647677 28544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:11.648649 28544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:11.649863 28544 hybrid_clock.cc:648] HybridClock initialized: now 1786518071649813 us; error 39 us; skew 500 ppm
I20260812 07:01:11.650215 28544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:01:11.652633 28544 webserver.cc:460] Webserver started at http://127.26.226.1:38915/ using document root <none> and password file <none>
I20260812 07:01:11.653340 28544 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:11.653463 28544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:11.653808 28544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:11.656922 28544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/data/instance:
uuid: "0052b7e1b52f4d2b852b532ed4fea9d2"
format_stamp: "Formatted at 2026-08-12 07:01:11 on dist-test-slave-t3q3"
server_key: "94aa9ca9f3cdf2be9be20ee2c6fea8ad"
server_key_iv: "7a529cfacf654004461e68b4e7c7e5af"
server_key_version: "encryptionkey@0"
I20260812 07:01:11.657776 28544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance:
uuid: "0052b7e1b52f4d2b852b532ed4fea9d2"
format_stamp: "Formatted at 2026-08-12 07:01:11 on dist-test-slave-t3q3"
server_key: "94aa9ca9f3cdf2be9be20ee2c6fea8ad"
server_key_iv: "7a529cfacf654004461e68b4e7c7e5af"
server_key_version: "encryptionkey@0"
I20260812 07:01:11.662959 28544 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:01:11.666481 28566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:11.667791 28544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:01:11.667977 28544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/wal
uuid: "0052b7e1b52f4d2b852b532ed4fea9d2"
format_stamp: "Formatted at 2026-08-12 07:01:11 on dist-test-slave-t3q3"
server_key: "94aa9ca9f3cdf2be9be20ee2c6fea8ad"
server_key_iv: "7a529cfacf654004461e68b4e7c7e5af"
server_key_version: "encryptionkey@0"
I20260812 07:01:11.668269 28544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:11.683357 28544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:11.684244 28544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:11.684546 28544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:11.685520 28544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:11.687150 28544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:11.687264 28544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:11.687402 28544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:11.687536 28544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:11.721498 28544 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.1:41881
I20260812 07:01:11.721580 28756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.1:41881 every 8 connection(s)
I20260812 07:01:11.722880 28544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/data/info.pb
I20260812 07:01:11.729831 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 28544
I20260812 07:01:11.730023 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-0/wal/instance
I20260812 07:01:11.730458 27528 external_mini_cluster.cc:1428] Setting key be80b683d9e7d894b1c824c8ecd48287
I20260812 07:01:11.733500 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.2:0
--local_ip_for_outbound_sockets=127.26.226.2
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33579
--builtin_ntp_servers=127.26.226.20:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:01:11.736326 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:11.725557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.1:33171 (local address 127.26.226.62:33579)
0812 07:01:11.726014 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:11.726019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:11.727112 (+ 1093us) server_negotiation.cc:407] Connection header received
0812 07:01:11.729101 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:11.729107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:11.729194 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:11.729309 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:11.730467 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:11.731399 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:11.733221 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:11.733683 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:11.733823 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:11.735520 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:11.735525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:11.735528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:11.735546 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:11.735593 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:11.735605 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:11.735673 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:11.735944 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 07:01:11.736019 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":252,"threads_started":1}
I20260812 07:01:11.737815 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:11.726089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33579 (local address 127.26.226.1:33171)
0812 07:01:11.726953 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:11.726997 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:11.728824 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:11.729549 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:11.729557 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:11.729640 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:11.730297 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:11.730316 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:11.731567 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:11.731571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:11.733072 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:11.733086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:11.733949 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:11.733959 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:11.734304 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:11.735130 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:01:11.735137 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:11.735197 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:01:11.735307 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:11.735705 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:11.735719 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:11.735730 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:01:11.735930 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 07:01:11.736382 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":274,"threads_started":1}
I20260812 07:01:11.739337 28760 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33579
I20260812 07:01:11.739641 28760 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:11.740483 28760 heartbeater.cc:511] Master 127.26.226.62:33579 requested a full tablet report, sending...
I20260812 07:01:11.742966 28441 ts_manager.cc:194] Registered new tserver with Master: 0052b7e1b52f4d2b852b532ed4fea9d2 (127.26.226.1:41881)
I20260812 07:01:11.744535 28441 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.1:33171
W20260812 07:01:12.092599 28764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:12.092891 28764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:12.093072 28764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:12.100772 28764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:12.100888 28764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:12.101058 28764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.2
I20260812 07:01:12.110729 28764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.226.2
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.2
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:12.112177 28764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:12.114109 28764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:12.128026 28769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:12.127995 28772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:12.128582 28776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:12.128867 28764 server_base.cc:1034] running on GCE node
I20260812 07:01:12.129467 28764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:12.130404 28764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:12.131608 28764 hybrid_clock.cc:648] HybridClock initialized: now 1786518072131561 us; error 43 us; skew 500 ppm
I20260812 07:01:12.131958 28764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:01:12.134378 28764 webserver.cc:460] Webserver started at http://127.26.226.2:40543/ using document root <none> and password file <none>
I20260812 07:01:12.135130 28764 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:12.135258 28764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:12.135604 28764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:12.138667 28764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/data/instance:
uuid: "4299a04c4c694af6a634494a648b28b0"
format_stamp: "Formatted at 2026-08-12 07:01:12 on dist-test-slave-t3q3"
server_key: "7e556265c7439f711af7dd4f481bfae4"
server_key_iv: "c7d8525f82ed975d56c6b2af3e773223"
server_key_version: "encryptionkey@0"
I20260812 07:01:12.139503 28764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance:
uuid: "4299a04c4c694af6a634494a648b28b0"
format_stamp: "Formatted at 2026-08-12 07:01:12 on dist-test-slave-t3q3"
server_key: "7e556265c7439f711af7dd4f481bfae4"
server_key_iv: "c7d8525f82ed975d56c6b2af3e773223"
server_key_version: "encryptionkey@0"
I20260812 07:01:12.145192 28764 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260812 07:01:12.148684 28791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:12.149991 28764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:01:12.150128 28764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/wal
uuid: "4299a04c4c694af6a634494a648b28b0"
format_stamp: "Formatted at 2026-08-12 07:01:12 on dist-test-slave-t3q3"
server_key: "7e556265c7439f711af7dd4f481bfae4"
server_key_iv: "c7d8525f82ed975d56c6b2af3e773223"
server_key_version: "encryptionkey@0"
I20260812 07:01:12.150316 28764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:12.175416 28764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:12.176319 28764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:12.176591 28764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:12.177524 28764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:12.178972 28764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:12.179060 28764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:12.179140 28764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:12.179252 28764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:12.214918 28764 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.2:46751
I20260812 07:01:12.215022 28967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.2:46751 every 8 connection(s)
I20260812 07:01:12.216406 28764 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/data/info.pb
I20260812 07:01:12.220340 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 28764
I20260812 07:01:12.220534 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/wal/instance
I20260812 07:01:12.221022 27528 external_mini_cluster.cc:1428] Setting key 547f484fed69b55b30ddf7656231d0ce
I20260812 07:01:12.224390 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.226.3:0
--local_ip_for_outbound_sockets=127.26.226.3
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33579
--builtin_ntp_servers=127.26.226.20:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:01:12.231976 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.219481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.2:50267 (local address 127.26.226.62:33579)
0812 07:01:12.219716 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:12.219721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:12.221300 (+ 1579us) server_negotiation.cc:407] Connection header received
0812 07:01:12.223591 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:12.223596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:12.223679 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:12.223808 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:12.225449 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.226659 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.228654 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.229397 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.229561 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.230942 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:12.230950 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:12.230958 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:12.230985 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:12.231031 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:12.231051 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:12.231146 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:12.231379 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 07:01:12.231577 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260812 07:01:12.232815 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.219980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33579 (local address 127.26.226.2:50267)
0812 07:01:12.221116 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:12.221172 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:12.223287 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:12.224034 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:12.224042 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:12.224208 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:12.225242 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.225262 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.226847 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.226851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.228487 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.228504 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.229641 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.229646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.229854 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.230595 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:01:12.230599 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:12.230641 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:01:12.230726 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:12.231164 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:12.231173 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:12.231179 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:01:12.231317 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 07:01:12.231613 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":350,"threads_started":1}
I20260812 07:01:12.234442 28968 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33579
I20260812 07:01:12.234754 28968 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:12.235488 28968 heartbeater.cc:511] Master 127.26.226.62:33579 requested a full tablet report, sending...
I20260812 07:01:12.237104 28441 ts_manager.cc:194] Registered new tserver with Master: 4299a04c4c694af6a634494a648b28b0 (127.26.226.2:46751)
I20260812 07:01:12.238408 28441 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.2:50267
W20260812 07:01:12.589581 28973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:12.589859 28973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:12.590009 28973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:12.597162 28973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:12.597265 28973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:12.597429 28973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.226.3
I20260812 07:01:12.606643 28973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.226.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.226.3
--webserver_port=0
--tserver_master_addrs=127.26.226.62:33579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.226.3
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:12.607879 28973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:12.609669 28973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:12.624742 28994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:12.624701 28985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:12.624907 28984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:12.625552 28973 server_base.cc:1034] running on GCE node
I20260812 07:01:12.626242 28973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:12.627177 28973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:12.628414 28973 hybrid_clock.cc:648] HybridClock initialized: now 1786518072628370 us; error 55 us; skew 500 ppm
I20260812 07:01:12.628727 28973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:01:12.631006 28973 webserver.cc:460] Webserver started at http://127.26.226.3:46135/ using document root <none> and password file <none>
I20260812 07:01:12.631705 28973 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:12.631836 28973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:12.632236 28973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:12.635046 28973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/data/instance:
uuid: "882d76d0defe4665b459786825a6d8bf"
format_stamp: "Formatted at 2026-08-12 07:01:12 on dist-test-slave-t3q3"
server_key: "5159e20df52320dafc9eb4ad97423d4d"
server_key_iv: "168340bb2b289e57d37618f42b8e5ee9"
server_key_version: "encryptionkey@0"
I20260812 07:01:12.635919 28973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance:
uuid: "882d76d0defe4665b459786825a6d8bf"
format_stamp: "Formatted at 2026-08-12 07:01:12 on dist-test-slave-t3q3"
server_key: "5159e20df52320dafc9eb4ad97423d4d"
server_key_iv: "168340bb2b289e57d37618f42b8e5ee9"
server_key_version: "encryptionkey@0"
I20260812 07:01:12.640944 28973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 07:01:12.644413 29000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:12.645663 28973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:01:12.645845 28973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/wal
uuid: "882d76d0defe4665b459786825a6d8bf"
format_stamp: "Formatted at 2026-08-12 07:01:12 on dist-test-slave-t3q3"
server_key: "5159e20df52320dafc9eb4ad97423d4d"
server_key_iv: "168340bb2b289e57d37618f42b8e5ee9"
server_key_version: "encryptionkey@0"
I20260812 07:01:12.646059 28973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:12.662778 28973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:12.663679 28973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:12.664005 28973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:12.665030 28973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:01:12.666501 28973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:01:12.666628 28973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:12.666764 28973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:01:12.666852 28973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:12.702087 28973 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.3:40953
I20260812 07:01:12.702221 29192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.3:40953 every 8 connection(s)
I20260812 07:01:12.703454 28973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/data/info.pb
I20260812 07:01:12.712813 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 28973
I20260812 07:01:12.713054 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/wal/instance
I20260812 07:01:12.713614 27528 external_mini_cluster.cc:1428] Setting key 7b73c827df090af0d6b49e87bd681767
I20260812 07:01:12.717192 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.706373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:56857 (local address 127.26.226.62:33579)
0812 07:01:12.706556 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:12.706561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:12.707981 (+ 1420us) server_negotiation.cc:407] Connection header received
0812 07:01:12.709788 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:12.709792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:12.709869 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:12.709968 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:12.711030 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.711859 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.713672 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.714311 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.714486 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.716308 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:12.716313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:12.716316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:12.716334 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:12.716369 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:12.716382 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:12.716501 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:12.716755 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 07:01:12.716860 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:01:12.718379 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.706921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:33579 (local address 127.26.226.3:56857)
0812 07:01:12.707807 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:12.707850 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:12.709525 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:12.710174 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:12.710195 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:12.710284 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:12.710848 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.710863 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.712000 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.712004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.713502 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.713516 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.714589 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.714599 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.714888 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.715820 (+ 932us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:01:12.715824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:12.715883 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:01:12.716021 (+ 138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:12.716489 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:12.716502 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:12.716512 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 07:01:12.716727 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 07:01:12.717189 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":281,"threads_started":1}
I20260812 07:01:12.719827 29193 heartbeater.cc:347] Connected to a master server at 127.26.226.62:33579
I20260812 07:01:12.720252 29193 heartbeater.cc:464] Registering TS with master...
I20260812 07:01:12.721081 29193 heartbeater.cc:511] Master 127.26.226.62:33579 requested a full tablet report, sending...
I20260812 07:01:12.722680 28441 ts_manager.cc:194] Registered new tserver with Master: 882d76d0defe4665b459786825a6d8bf (127.26.226.3:40953)
I20260812 07:01:12.723601 28441 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.226.3:56857
I20260812 07:01:12.729449 27528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:01:12.741777 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.735674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.26.226.62:33579)
0812 07:01:12.735951 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:12.735959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:12.736317 (+ 358us) server_negotiation.cc:407] Connection header received
0812 07:01:12.736530 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:12.736535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:12.736624 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:12.736740 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:12.737251 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.737972 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.739378 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.739808 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.739950 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.741037 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:12.741049 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:12.741053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:12.741071 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:12.741101 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:12.741113 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:12.741209 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:12.741369 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 07:01:12.741463 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 07:01:12.747634 28760 heartbeater.cc:503] Master 127.26.226.62:33579 was elected leader, sending a full tablet report...
I20260812 07:01:12.745832 28441 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:01:12.753345 28441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:01:12.779196 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.765795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46174 (local address 127.26.226.2:46751)
0812 07:01:12.766476 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:12.766487 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:12.767340 (+ 853us) server_negotiation.cc:407] Connection header received
0812 07:01:12.767799 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:12.767811 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:12.768176 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:12.768367 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:12.769561 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.771505 (+ 1944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.775166 (+ 3661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.777225 (+ 2059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.777375 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.777707 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:12.778589 (+ 882us) server_negotiation.cc:299] Negotiation successful
0812 07:01:12.778787 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":2027,"threads_started":1}
I20260812 07:01:12.780218 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.765540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:46751 (local address 127.0.0.1:46174)
0812 07:01:12.767152 (+ 1612us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:12.767209 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:12.767520 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:12.768783 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:12.768794 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:12.768833 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:12.769343 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.769359 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.771746 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.771754 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.774850 (+ 3096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.774874 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.777971 (+ 3097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.777982 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.778272 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.778335 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 07:01:12.779441 (+ 1106us) client_negotiation.cc:240] Negotiation successful
0812 07:01:12.779760 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1265,"thread_start_us":13557,"threads_started":1}
I20260812 07:01:12.784653 28863 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa181a9308f48e488a87111afaa1bb0 (DEFAULT_TABLE table=test-table [id=a80bc8f5034a4572bbf60336c9de2b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:12.786247 28863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa181a9308f48e488a87111afaa1bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:12.787452 29210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.766371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.0.0.1:56050)
0812 07:01:12.779491 (+ 13120us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:12.779508 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:12.779725 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:12.780563 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:12.780571 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:12.780615 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:12.780961 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.780974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.782991 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.782997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.784758 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.784771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.786586 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.786593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.786797 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.786835 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 07:01:12.786945 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 07:01:12.787076 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12862,"mutex_wait_us":102,"thread_start_us":745,"threads_started":1}
I20260812 07:01:12.787504 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.766961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.26.226.1:41881)
0812 07:01:12.777579 (+ 10618us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:12.777593 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:12.779615 (+ 2022us) server_negotiation.cc:407] Connection header received
0812 07:01:12.779912 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:12.779925 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:12.780276 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:12.780503 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:12.781197 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.782852 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.784951 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.786396 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.786520 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.786808 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:12.787001 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 07:01:12.787155 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10347,"thread_start_us":10640,"threads_started":1}
I20260812 07:01:12.788677 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.765861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.3:40953 (local address 127.0.0.1:48546)
0812 07:01:12.778445 (+ 12584us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:12.778473 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:12.778656 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:12.780967 (+ 2311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:12.780971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:12.780996 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:12.781356 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.781395 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.783334 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.783339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.785602 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.785618 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.787879 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.787892 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.788090 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.788126 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:01:12.788215 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 07:01:12.788374 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12384,"thread_start_us":12185,"threads_started":1}
I20260812 07:01:12.789099 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.766548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.26.226.3:40953)
0812 07:01:12.780399 (+ 13851us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:12.780410 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:12.780446 (+ 36us) server_negotiation.cc:407] Connection header received
0812 07:01:12.780542 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:12.780551 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:12.780783 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:12.780974 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:12.781599 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.783156 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.785867 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.787326 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.788177 (+ 851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.788465 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:12.788542 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 07:01:12.788730 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13645,"thread_start_us":13439,"threads_started":1}
I20260812 07:01:12.791095 28648 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa181a9308f48e488a87111afaa1bb0 (DEFAULT_TABLE table=test-table [id=a80bc8f5034a4572bbf60336c9de2b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:12.792356 28648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa181a9308f48e488a87111afaa1bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:12.792706 29080 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa181a9308f48e488a87111afaa1bb0 (DEFAULT_TABLE table=test-table [id=a80bc8f5034a4572bbf60336c9de2b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:01:12.794195 29080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa181a9308f48e488a87111afaa1bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:12.804771 29222 tablet_bootstrap.cc:492] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: Bootstrap starting.
I20260812 07:01:12.809001 29223 tablet_bootstrap.cc:492] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2: Bootstrap starting.
I20260812 07:01:12.809604 29222 tablet_bootstrap.cc:654] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:12.811085 29222 log.cc:824] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:12.813297 29223 tablet_bootstrap.cc:654] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:12.814159 29222 tablet_bootstrap.cc:492] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: No bootstrap required, opened a new log
I20260812 07:01:12.814203 29223 log.cc:824] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:12.814463 29222 ts_tablet_manager.cc:1397] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260812 07:01:12.815153 29224 tablet_bootstrap.cc:492] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: Bootstrap starting.
I20260812 07:01:12.817349 29223 tablet_bootstrap.cc:492] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2: No bootstrap required, opened a new log
I20260812 07:01:12.817685 29223 ts_tablet_manager.cc:1397] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 07:01:12.818338 29224 tablet_bootstrap.cc:654] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:12.819442 29224 log.cc:824] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:12.822294 29224 tablet_bootstrap.cc:492] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: No bootstrap required, opened a new log
I20260812 07:01:12.822567 29224 ts_tablet_manager.cc:1397] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.002s
I20260812 07:01:12.825019 29222 raft_consensus.cc:348] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 } } peers { permanent_uuid: "882d76d0defe4665b459786825a6d8bf" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 40953 } } peers { permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 } }
I20260812 07:01:12.825479 29222 raft_consensus.cc:374] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:12.825580 29222 raft_consensus.cc:729] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4299a04c4c694af6a634494a648b28b0, State: Initialized, Role: FOLLOWER
I20260812 07:01:12.826229 29222 consensus_queue.cc:260] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 } } peers { permanent_uuid: "882d76d0defe4665b459786825a6d8bf" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 40953 } } peers { permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 } }
I20260812 07:01:12.827445 28968 heartbeater.cc:503] Master 127.26.226.62:33579 was elected leader, sending a full tablet report...
I20260812 07:01:12.828119 29222 ts_tablet_manager.cc:1428] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260812 07:01:12.828903 29223 raft_consensus.cc:348] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 } } peers { permanent_uuid: "882d76d0defe4665b459786825a6d8bf" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 40953 } } peers { permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 } }
I20260812 07:01:12.829325 29223 raft_consensus.cc:374] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:12.829442 29223 raft_consensus.cc:729] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0052b7e1b52f4d2b852b532ed4fea9d2, State: Initialized, Role: FOLLOWER
I20260812 07:01:12.830030 29223 consensus_queue.cc:260] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 } } peers { permanent_uuid: "882d76d0defe4665b459786825a6d8bf" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 40953 } } peers { permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 } }
I20260812 07:01:12.831856 29223 ts_tablet_manager.cc:1428] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260812 07:01:12.831795 29224 raft_consensus.cc:348] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 } } peers { permanent_uuid: "882d76d0defe4665b459786825a6d8bf" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 40953 } } peers { permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 } }
I20260812 07:01:12.832315 29224 raft_consensus.cc:374] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:12.832420 29224 raft_consensus.cc:729] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882d76d0defe4665b459786825a6d8bf, State: Initialized, Role: FOLLOWER
I20260812 07:01:12.833014 29224 consensus_queue.cc:260] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 } } peers { permanent_uuid: "882d76d0defe4665b459786825a6d8bf" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 40953 } } peers { permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 } }
I20260812 07:01:12.834213 29193 heartbeater.cc:503] Master 127.26.226.62:33579 was elected leader, sending a full tablet report...
I20260812 07:01:12.834864 29224 ts_tablet_manager.cc:1428] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: Time spent starting tablet: real 0.012s user 0.005s sys 0.007s
I20260812 07:01:12.922883 29234 raft_consensus.cc:482] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:01:12.923321 29234 raft_consensus.cc:504] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 } } peers { permanent_uuid: "882d76d0defe4665b459786825a6d8bf" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 40953 } } peers { permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 } }
I20260812 07:01:12.925390 29234 leader_election.cc:288] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4299a04c4c694af6a634494a648b28b0 (127.26.226.2:46751), 0052b7e1b52f4d2b852b532ed4fea9d2 (127.26.226.1:41881)
I20260812 07:01:12.932729 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.925935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:48123 (local address 127.26.226.1:41881)
0812 07:01:12.926175 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:12.926199 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:12.926274 (+ 75us) server_negotiation.cc:407] Connection header received
0812 07:01:12.926502 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:12.926511 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:12.926634 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:12.926746 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:12.927296 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.928211 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.930699 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.931730 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.931896 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.931969 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:12.932339 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 07:01:12.932475 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 07:01:12.932742 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.925868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.26.226.3:48123)
0812 07:01:12.926153 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:12.926178 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:12.926338 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:12.926837 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:12.926843 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:12.926875 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:12.927109 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.927125 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.928365 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.928369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.930503 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.930546 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.931889 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.931893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.932173 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.932201 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 07:01:12.932285 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 07:01:12.932394 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 07:01:12.933632 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.925887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.226.3:32851 (local address 127.26.226.2:46751)
0812 07:01:12.926174 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:12.926182 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:12.927153 (+ 971us) server_negotiation.cc:407] Connection header received
0812 07:01:12.927336 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:12.927343 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:12.927437 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:12.927537 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:01:12.928050 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.928845 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.931261 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.932477 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.932628 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.932700 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:12.933208 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 07:01:12.933337 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260812 07:01:12.934049 28682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa181a9308f48e488a87111afaa1bb0" candidate_uuid: "882d76d0defe4665b459786825a6d8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" is_pre_election: true
I20260812 07:01:12.934386 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.925971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.2:46751 (local address 127.26.226.3:32851)
0812 07:01:12.927020 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:12.927057 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:12.927198 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:12.927561 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:12.927564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:12.927585 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:01:12.927906 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.927913 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.929018 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.929023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.931080 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:12.931092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.932660 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.932666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:12.932887 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.932929 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 07:01:12.933838 (+ 909us) client_negotiation.cc:240] Negotiation successful
0812 07:01:12.933987 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":179,"thread_start_us":456,"threads_started":1}
I20260812 07:01:12.934795 28682 raft_consensus.cc:2454] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 882d76d0defe4665b459786825a6d8bf in term 0.
I20260812 07:01:12.935506 28903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa181a9308f48e488a87111afaa1bb0" candidate_uuid: "882d76d0defe4665b459786825a6d8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4299a04c4c694af6a634494a648b28b0" is_pre_election: true
I20260812 07:01:12.935609 29002 leader_election.cc:302] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0052b7e1b52f4d2b852b532ed4fea9d2, 882d76d0defe4665b459786825a6d8bf; no voters:
I20260812 07:01:12.935998 29234 raft_consensus.cc:2781] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:01:12.935997 28903 raft_consensus.cc:2454] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 882d76d0defe4665b459786825a6d8bf in term 0.
I20260812 07:01:12.936221 29234 raft_consensus.cc:482] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:01:12.936329 29234 raft_consensus.cc:3037] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:12.938664 29234 raft_consensus.cc:504] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 } } peers { permanent_uuid: "882d76d0defe4665b459786825a6d8bf" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 40953 } } peers { permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 } }
I20260812 07:01:12.939325 29234 leader_election.cc:288] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [CANDIDATE]: Term 1 election: Requested vote from peers 4299a04c4c694af6a634494a648b28b0 (127.26.226.2:46751), 0052b7e1b52f4d2b852b532ed4fea9d2 (127.26.226.1:41881)
I20260812 07:01:12.939716 28903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa181a9308f48e488a87111afaa1bb0" candidate_uuid: "882d76d0defe4665b459786825a6d8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4299a04c4c694af6a634494a648b28b0"
I20260812 07:01:12.939911 28903 raft_consensus.cc:3037] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:12.939927 28682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa181a9308f48e488a87111afaa1bb0" candidate_uuid: "882d76d0defe4665b459786825a6d8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2"
I20260812 07:01:12.940176 28682 raft_consensus.cc:3037] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:12.942245 28903 raft_consensus.cc:2454] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 882d76d0defe4665b459786825a6d8bf in term 1.
I20260812 07:01:12.942639 29005 leader_election.cc:302] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4299a04c4c694af6a634494a648b28b0, 882d76d0defe4665b459786825a6d8bf; no voters:
I20260812 07:01:12.942723 28682 raft_consensus.cc:2454] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 882d76d0defe4665b459786825a6d8bf in term 1.
I20260812 07:01:12.942955 29234 raft_consensus.cc:2781] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:12.943399 29234 raft_consensus.cc:686] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 1 LEADER]: Becoming Leader. State: Replica: 882d76d0defe4665b459786825a6d8bf, State: Running, Role: LEADER
I20260812 07:01:12.943898 29234 consensus_queue.cc:237] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 } } peers { permanent_uuid: "882d76d0defe4665b459786825a6d8bf" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 40953 } } peers { permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 } }
I20260812 07:01:12.948952 28441 catalog_manager.cc:5495] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 882d76d0defe4665b459786825a6d8bf (127.26.226.3). New cstate: current_term: 1 leader_uuid: "882d76d0defe4665b459786825a6d8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "882d76d0defe4665b459786825a6d8bf" member_type: VOTER last_known_addr { host: "127.26.226.3" port: 40953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 } health_report { overall_health: UNKNOWN } } }
W20260812 07:01:12.955184 29194 tablet.cc:2369] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:01:12.968547 28971 tablet.cc:2369] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:01:12.975339 28761 tablet.cc:2369] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:01:12.983973 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:12.978931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.26.226.3:40953)
0812 07:01:12.979124 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:12.979128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:12.979146 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:01:12.979280 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:12.979283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:12.979370 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:12.979498 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:01:12.979810 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.980617 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.981933 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:12.982353 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:12.982447 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:12.982882 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:01:12.983381 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:01:12.983490 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:12.983621 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 07:01:12.983683 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:01:12.992687 28903 raft_consensus.cc:1261] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0 [term 1 FOLLOWER]: Refusing update from remote peer 882d76d0defe4665b459786825a6d8bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:01:12.992687 28682 raft_consensus.cc:1261] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2 [term 1 FOLLOWER]: Refusing update from remote peer 882d76d0defe4665b459786825a6d8bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:01:12.993924 29236 consensus_queue.cc:1035] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0052b7e1b52f4d2b852b532ed4fea9d2" member_type: VOTER last_known_addr { host: "127.26.226.1" port: 41881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:01:12.994354 29234 consensus_queue.cc:1035] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4299a04c4c694af6a634494a648b28b0" member_type: VOTER last_known_addr { host: "127.26.226.2" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:01:13.020756 28440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:01:13.021062 28440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:01:13.024799 28440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c14f1be78344bab80868fd15907e565: Sending DeleteTablet for 3 replicas of tablet 2fa181a9308f48e488a87111afaa1bb0
I20260812 07:01:13.025985 29080 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa181a9308f48e488a87111afaa1bb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:13 UTC) from {username='slave'} at 127.0.0.1:48546
I20260812 07:01:13.026006 28863 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa181a9308f48e488a87111afaa1bb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:13 UTC) from {username='slave'} at 127.0.0.1:46174
I20260812 07:01:13.026803 28648 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa181a9308f48e488a87111afaa1bb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:01:13 UTC) from {username='slave'} at 127.0.0.1:56050
I20260812 07:01:13.027076 29256 tablet_replica.cc:330] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: stopping tablet replica
I20260812 07:01:13.027585 29256 raft_consensus.cc:2229] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:01:13.028420 29256 raft_consensus.cc:2258] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:13.029155 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 28544
I20260812 07:01:13.030992 29256 ts_tablet_manager.cc:1905] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:13.040409 29256 ts_tablet_manager.cc:1918] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:01:13.040755 29256 log.cc:1196] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-2/wal/wals/2fa181a9308f48e488a87111afaa1bb0
I20260812 07:01:13.041520 29256 ts_tablet_manager.cc:1939] T 2fa181a9308f48e488a87111afaa1bb0 P 882d76d0defe4665b459786825a6d8bf: Deleting consensus metadata
I20260812 07:01:13.043957 28417 catalog_manager.cc:4841] TS 882d76d0defe4665b459786825a6d8bf (127.26.226.3:40953): tablet 2fa181a9308f48e488a87111afaa1bb0 (table test-table [id=a80bc8f5034a4572bbf60336c9de2b4f]) successfully deleted
I20260812 07:01:13.045499 29260 tablet_replica.cc:330] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2: stopping tablet replica
I20260812 07:01:13.046146 29260 raft_consensus.cc:2229] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:01:13.046875 29260 raft_consensus.cc:2258] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:13.047822 29258 tablet_replica.cc:330] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: stopping tablet replica
I20260812 07:01:13.048352 29258 raft_consensus.cc:2229] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:01:13.048924 29258 raft_consensus.cc:2258] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:01:13.050091 29260 ts_tablet_manager.cc:1905] T 2fa181a9308f48e488a87111afaa1bb0 P 0052b7e1b52f4d2b852b532ed4fea9d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:13.051152 29258 ts_tablet_manager.cc:1905] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:01:13.054414 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.030742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.26.226.1:41881)
0812 07:01:13.030923 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:13.030930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:13.044776 (+ 13846us) server_negotiation.cc:407] Connection header received
0812 07:01:13.045004 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:13.045015 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:13.045173 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:13.045302 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:13.046108 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:13.047300 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:13.049693 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:13.050350 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:13.050456 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:13.052222 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:13.052233 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:13.052250 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:13.053378 (+ 1128us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:13.053466 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:13.053492 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:13.053581 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:13.053903 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 07:01:13.054025 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:01:13.055835 28752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:13.058872 29258 ts_tablet_manager.cc:1918] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:01:13.059070 29258 log.cc:1196] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: Deleting WAL directory at /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517734245872-27528-0/minicluster-data/ts-1/wal/wals/2fa181a9308f48e488a87111afaa1bb0
I20260812 07:01:13.059654 29258 ts_tablet_manager.cc:1939] T 2fa181a9308f48e488a87111afaa1bb0 P 4299a04c4c694af6a634494a648b28b0: Deleting consensus metadata
I20260812 07:01:13.061486 28425 catalog_manager.cc:4841] TS 4299a04c4c694af6a634494a648b28b0 (127.26.226.2:46751): tablet 2fa181a9308f48e488a87111afaa1bb0 (table test-table [id=a80bc8f5034a4572bbf60336c9de2b4f]) successfully deleted
I20260812 07:01:13.150841 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 28544
W20260812 07:01:13.176594 28418 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.226.1:41881 (error 108)
I20260812 07:01:13.176707 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 28764
I20260812 07:01:13.179322 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.178740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.0.0.1:56064)
0812 07:01:13.178925 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:13.179137 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 07:01:13.179741 28418 catalog_manager.cc:4584] TS 0052b7e1b52f4d2b852b532ed4fea9d2 (127.26.226.1:41881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2fa181a9308f48e488a87111afaa1bb0: Network error: Client connection negotiation failed: client connection to 127.26.226.1:41881: connect: Connection refused (error 111)
I20260812 07:01:13.184911 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.178137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46180 (local address 127.26.226.2:46751)
0812 07:01:13.178313 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:13.178322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:13.178344 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:01:13.178410 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:13.178415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:13.178525 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:13.178636 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:13.179485 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:13.180511 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:13.181772 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:13.182192 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:13.182332 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:13.183431 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:13.183440 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:13.183454 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:13.184027 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:13.184131 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:13.184154 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:13.184294 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:13.184434 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 07:01:13.184546 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:01:13.185642 28960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:13.233740 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.233173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.0.0.1:56070)
0812 07:01:13.233421 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:13.233535 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 07:01:13.234951 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.234522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.0.0.1:56086)
0812 07:01:13.234705 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:13.234796 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:01:13.278443 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 28764
I20260812 07:01:13.295683 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.295120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.0.0.1:56100)
0812 07:01:13.295354 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:13.295466 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 07:01:13.296978 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.296523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.0.0.1:56114)
0812 07:01:13.296718 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:13.296815 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:01:13.306432 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 28973
I20260812 07:01:13.315078 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.307328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48568 (local address 127.26.226.3:40953)
0812 07:01:13.307529 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:13.307535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:13.307561 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:01:13.307724 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:13.307730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:13.307851 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:13.308003 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:13.308474 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:13.309707 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:13.311094 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:13.311833 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:13.311984 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:13.313323 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:13.313335 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:13.313355 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:13.314222 (+ 867us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:13.314285 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:13.314312 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:13.314437 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:13.314585 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 07:01:13.314696 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 07:01:13.315917 29184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:13.377719 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.377169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.0.0.1:56130)
0812 07:01:13.377438 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:13.377532 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260812 07:01:13.378824 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.378425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.0.0.1:56136)
0812 07:01:13.378629 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:13.378713 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260812 07:01:13.438450 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 28973
I20260812 07:01:13.467484 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 28399
I20260812 07:01:13.468731 28489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:13.580957 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 28399
I20260812 07:01:13.581004 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.580463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.0.0.1:56144)
0812 07:01:13.580692 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:13.580801 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 07:01:13.582046 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:13.581685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.1:41881 (local address 127.0.0.1:56160)
0812 07:01:13.581861 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:13.581939 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.1:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
2026-08-12T07:01:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2997 ms)
[----------] 6 tests from EncryptionPolicyTest (24845 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:13 dist-test-slave-t3q3 krb5kdc[29276](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:13 dist-test-slave-t3q3 krb5kdc[29276](info): set up 2 sockets
Aug 12 07:01:13 dist-test-slave-t3q3 krb5kdc[29276](info): commencing operation
krb5kdc: starting...
W20260812 07:01:14.999120 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.359s user 0.002s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:15 dist-test-slave-t3q3 krb5kdc[29276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:15.357800 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:15.722771 29300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:15.723202 29300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:15.723377 29300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:15.730885 29300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:15.731070 29300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:15.731182 29300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:15.731295 29300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:15.731374 29300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:15.741770 29300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37161
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:15.743268 29300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:15.744694 29300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:15.757107 29306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:15.757135 29308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:15.757194 29311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:15.758088 29300 server_base.cc:1034] running on GCE node
I20260812 07:01:15.758858 29300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:15.760360 29300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:15.761586 29300 hybrid_clock.cc:648] HybridClock initialized: now 1786518075761549 us; error 32 us; skew 500 ppm
I20260812 07:01:15.761994 29300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:15 dist-test-slave-t3q3 krb5kdc[29276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518075, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:15.768903 29300 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:15.770465 29300 webserver.cc:460] Webserver started at http://127.0.0.1:36463/ using document root <none> and password file <none>
I20260812 07:01:15.771173 29300 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:15.771283 29300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:15.771538 29300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:15.774807 29300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "f9047a4d08c24332a751e72f3299bd8a"
format_stamp: "Formatted at 2026-08-12 07:01:15 on dist-test-slave-t3q3"
server_key: "4683f94498cc75470492027754e3289b"
server_key_iv: "d90506c399da8f77e1e7d5c365066ece"
server_key_version: "encryptionkey@0"
I20260812 07:01:15.775544 29300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "f9047a4d08c24332a751e72f3299bd8a"
format_stamp: "Formatted at 2026-08-12 07:01:15 on dist-test-slave-t3q3"
server_key: "4683f94498cc75470492027754e3289b"
server_key_iv: "d90506c399da8f77e1e7d5c365066ece"
server_key_version: "encryptionkey@0"
I20260812 07:01:15.780704 29300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 07:01:15.784226 29322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:15.785528 29300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:01:15.785732 29300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "f9047a4d08c24332a751e72f3299bd8a"
format_stamp: "Formatted at 2026-08-12 07:01:15 on dist-test-slave-t3q3"
server_key: "4683f94498cc75470492027754e3289b"
server_key_iv: "d90506c399da8f77e1e7d5c365066ece"
server_key_version: "encryptionkey@0"
I20260812 07:01:15.785928 29300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:15.805969 29300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:15.810825 29300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:15.811172 29300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:15.831097 29300 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37161
I20260812 07:01:15.831151 29421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37161 every 8 connection(s)
I20260812 07:01:15.832670 29300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:15.833330 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 29300
I20260812 07:01:15.833616 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:15.834448 27528 external_mini_cluster.cc:1428] Setting key 6ca9d36eb2e65f6d2eb8285d7ec902b1
I20260812 07:01:15.837450 29427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 07:01:15 dist-test-slave-t3q3 krb5kdc[29276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:01:15.848989 29427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a: Bootstrap starting.
I20260812 07:01:15.852686 29427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:15.853801 29427 log.cc:824] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:15.856879 29427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a: No bootstrap required, opened a new log
I20260812 07:01:15.857801 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:15.836618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47403 (local address 127.0.0.1:37161)
0812 07:01:15.837583 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:15.837604 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:15.837646 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:01:15.839286 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:15.839307 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:15.839668 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:15.840061 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:15.841644 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:15.842978 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:15.844332 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:15.844999 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:15.845210 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:15.849818 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:15.849827 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:15.849841 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:15.849883 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:15.853375 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:15.854318 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:15.854325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:15.854332 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:15.854521 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:15.855052 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:15.855056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:15.855058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:15.855558 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:15.855905 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:15.856431 (+ 526us) server_negotiation.cc:299] Negotiation successful
0812 07:01:15.856749 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":443,"threads_started":1}
I20260812 07:01:15.862038 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:15.865461 29427 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9047a4d08c24332a751e72f3299bd8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } }
I20260812 07:01:15.865839 29427 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:15.865984 29427 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9047a4d08c24332a751e72f3299bd8a, State: Initialized, Role: FOLLOWER
I20260812 07:01:15.866555 29427 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9047a4d08c24332a751e72f3299bd8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } }
I20260812 07:01:15.866811 29427 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:15.866955 29427 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:15.867182 29427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:15.870604 29427 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9047a4d08c24332a751e72f3299bd8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } }
I20260812 07:01:15.871150 29427 leader_election.cc:302] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9047a4d08c24332a751e72f3299bd8a; no voters:
I20260812 07:01:15.871842 29427 leader_election.cc:288] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:15.871956 29446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:15.872658 29446 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [term 1 LEADER]: Becoming Leader. State: Replica: f9047a4d08c24332a751e72f3299bd8a, State: Running, Role: LEADER
I20260812 07:01:15.873306 29446 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9047a4d08c24332a751e72f3299bd8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } }
I20260812 07:01:15.875115 29427 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:15.877415 29447 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9047a4d08c24332a751e72f3299bd8a. Latest consensus state: current_term: 1 leader_uuid: "f9047a4d08c24332a751e72f3299bd8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9047a4d08c24332a751e72f3299bd8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } } }
I20260812 07:01:15.877848 29447 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:15.877972 29448 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9047a4d08c24332a751e72f3299bd8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9047a4d08c24332a751e72f3299bd8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } } }
I20260812 07:01:15.878334 29448 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:15.889083 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:15.867290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58871 (local address 127.0.0.1:37161)
0812 07:01:15.867536 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:15.867545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:15.867983 (+ 438us) server_negotiation.cc:407] Connection header received
0812 07:01:15.868191 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:15.868194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:15.868284 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:15.868372 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:15.870250 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:15.871065 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:15.873261 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:15.873999 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:15.874136 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:15.878667 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:15.878677 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:15.878685 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:15.878736 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:15.883433 (+ 4697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:15.884842 (+ 1409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:15.884851 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:15.884858 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:15.885161 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:15.885928 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:15.885936 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:15.885942 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:15.886449 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:15.886847 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:15.888278 (+ 1431us) server_negotiation.cc:299] Negotiation successful
0812 07:01:15.888532 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:01:15.892184 29453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:15.898093 29453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:15.907478 29453 catalog_manager.cc:1347] Generated new cluster ID: be8200a9aae24ecba20c36ce553d981a
I20260812 07:01:15.907614 29453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:15.929762 29453 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:15.931133 29453 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:15.942349 29453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9047a4d08c24332a751e72f3299bd8a: Generated new TSK 0
I20260812 07:01:15.943212 29453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:15.952328 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29300
I20260812 07:01:15.953603 29413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:16.046573 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 29300
W20260812 07:01:16.074213 29270 connection.cc:535] client connection to 127.0.0.1:37161 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37161): Connection reset by peer (error 104)
2026-08-12T07:01:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2453 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:16 dist-test-slave-t3q3 krb5kdc[29480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:16 dist-test-slave-t3q3 krb5kdc[29480](info): set up 2 sockets
Aug 12 07:01:16 dist-test-slave-t3q3 krb5kdc[29480](info): commencing operation
krb5kdc: starting...
W20260812 07:01:17.443712 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:17 dist-test-slave-t3q3 krb5kdc[29480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:17.797534 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:18.158336 29495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:18.158633 29495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:18.158807 29495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:18.166090 29495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:18.166250 29495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:18.166337 29495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:18.166448 29495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:18.166500 29495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:18.176374 29495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:18.177696 29495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:18.179033 29495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:18.191367 29503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:18.191367 29504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:18.191385 29507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:18.192610 29495 server_base.cc:1034] running on GCE node
I20260812 07:01:18.193553 29495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:18.195102 29495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:18.196359 29495 hybrid_clock.cc:648] HybridClock initialized: now 1786518078196297 us; error 63 us; skew 500 ppm
I20260812 07:01:18.196792 29495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:18 dist-test-slave-t3q3 krb5kdc[29480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518078, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:18.200205 29495 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:18.201694 29495 webserver.cc:460] Webserver started at http://127.0.0.1:37385/ using document root <none> and password file <none>
I20260812 07:01:18.202380 29495 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:18.202486 29495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:18.202828 29495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:18.206022 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "513e8374d9994ad9afcc66003d50ce64"
format_stamp: "Formatted at 2026-08-12 07:01:18 on dist-test-slave-t3q3"
server_key: "922443fb742d2c118ba9821de0361e84"
server_key_iv: "baf7f91e260e9af8095652653d4d7333"
server_key_version: "encryptionkey@0"
I20260812 07:01:18.206804 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "513e8374d9994ad9afcc66003d50ce64"
format_stamp: "Formatted at 2026-08-12 07:01:18 on dist-test-slave-t3q3"
server_key: "922443fb742d2c118ba9821de0361e84"
server_key_iv: "baf7f91e260e9af8095652653d4d7333"
server_key_version: "encryptionkey@0"
I20260812 07:01:18.212416 29495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 07:01:18.216274 29517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:18.217664 29495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:01:18.217867 29495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "513e8374d9994ad9afcc66003d50ce64"
format_stamp: "Formatted at 2026-08-12 07:01:18 on dist-test-slave-t3q3"
server_key: "922443fb742d2c118ba9821de0361e84"
server_key_iv: "baf7f91e260e9af8095652653d4d7333"
server_key_version: "encryptionkey@0"
I20260812 07:01:18.218048 29495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:18.229431 29495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:18.233675 29495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:18.233994 29495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:18.253327 29495 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43067
I20260812 07:01:18.253358 29608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43067 every 8 connection(s)
I20260812 07:01:18.254796 29495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:18.258623 29610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:18.264283 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 29495
I20260812 07:01:18.264470 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:18.265054 27528 external_mini_cluster.cc:1428] Setting key b80e69d15e07063ba183a837ca1c34ae
I20260812 07:01:18.270340 29610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64: Bootstrap starting.
I20260812 07:01:18.274307 29610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:18.275575 29610 log.cc:824] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:18.278824 29610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64: No bootstrap required, opened a new log
I20260812 07:01:18.279429 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:18.266971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47689 (local address 127.0.0.1:43067)
0812 07:01:18.267911 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:18.267922 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:18.267969 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:01:18.269855 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:18.269880 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:18.270284 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:18.270676 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:18.271355 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:18.272770 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:18.274703 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:18.275394 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:18.275533 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:18.277064 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:18.277073 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:18.277089 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:18.277618 (+ 529us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:18.277694 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:18.277727 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:18.277914 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:18.278198 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 07:01:18.278389 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":478,"threads_started":1}
I20260812 07:01:18.283710 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:18.287462 29610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513e8374d9994ad9afcc66003d50ce64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260812 07:01:18.287864 29610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:18.287993 29610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513e8374d9994ad9afcc66003d50ce64, State: Initialized, Role: FOLLOWER
I20260812 07:01:18.288672 29610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513e8374d9994ad9afcc66003d50ce64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260812 07:01:18.288995 29610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:18.289160 29610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:18.289371 29610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:18.292568 29610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513e8374d9994ad9afcc66003d50ce64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260812 07:01:18.293200 29610 leader_election.cc:302] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 513e8374d9994ad9afcc66003d50ce64; no voters:
I20260812 07:01:18.293942 29610 leader_election.cc:288] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:18.294056 29632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:18.295644 29632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [term 1 LEADER]: Becoming Leader. State: Replica: 513e8374d9994ad9afcc66003d50ce64, State: Running, Role: LEADER
I20260812 07:01:18.296360 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:18.288651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59043 (local address 127.0.0.1:43067)
0812 07:01:18.288864 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:18.288870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:18.289272 (+ 402us) server_negotiation.cc:407] Connection header received
0812 07:01:18.289508 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:18.289514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:18.289604 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:18.289731 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:18.290268 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:18.291449 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:18.292814 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:18.293328 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:18.293484 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:18.294960 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:18.294966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:18.294971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:18.294993 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:18.295068 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:18.295090 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:18.295196 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:18.295880 (+ 684us) server_negotiation.cc:299] Negotiation successful
0812 07:01:18.295940 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:01:18.296466 29632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513e8374d9994ad9afcc66003d50ce64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260812 07:01:18.296860 29610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:18.299700 29635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 513e8374d9994ad9afcc66003d50ce64. Latest consensus state: current_term: 1 leader_uuid: "513e8374d9994ad9afcc66003d50ce64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513e8374d9994ad9afcc66003d50ce64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } } }
I20260812 07:01:18.300128 29634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "513e8374d9994ad9afcc66003d50ce64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513e8374d9994ad9afcc66003d50ce64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } } }
I20260812 07:01:18.300384 29635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:18.300441 29634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:18.302261 29646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:18.307421 29646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:18.316605 29646 catalog_manager.cc:1347] Generated new cluster ID: d83405c52ab04f119ed0ecf9c513acab
I20260812 07:01:18.316764 29646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:18.336416 29646 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:18.337846 29646 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:18.348675 29646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 513e8374d9994ad9afcc66003d50ce64: Generated new TSK 0
I20260812 07:01:18.349573 29646 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:18.362816 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29495
W20260812 07:01:18.364141 29601 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47689
W20260812 07:01:18.364976 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29495: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:18.365183 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 29495
W20260812 07:01:18.392544 29478 connection.cc:535] client connection to 127.0.0.1:43067 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43067): Connection reset by peer (error 104)
2026-08-12T07:01:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2318 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:18 dist-test-slave-t3q3 krb5kdc[29670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:18 dist-test-slave-t3q3 krb5kdc[29670](info): set up 2 sockets
Aug 12 07:01:18 dist-test-slave-t3q3 krb5kdc[29670](info): commencing operation
krb5kdc: starting...
W20260812 07:01:19.743870 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:19 dist-test-slave-t3q3 krb5kdc[29670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:20.093834 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:20.447741 29694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:20.448132 29694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:20.448318 29694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:20.455617 29694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:20.455778 29694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:20.455885 29694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:20.455996 29694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:20.456166 29694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:20.466533 29694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:20.468034 29694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:20.469476 29694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:20.481976 29704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:20.482053 29702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:20.482002 29700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:20.483156 29694 server_base.cc:1034] running on GCE node
I20260812 07:01:20.484004 29694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:20.485569 29694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:20.486800 29694 hybrid_clock.cc:648] HybridClock initialized: now 1786518080486741 us; error 40 us; skew 500 ppm
I20260812 07:01:20.487214 29694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:20 dist-test-slave-t3q3 krb5kdc[29670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518080, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:20.490643 29694 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:20.492411 29694 webserver.cc:460] Webserver started at http://127.0.0.1:41977/ using document root <none> and password file <none>
I20260812 07:01:20.493173 29694 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:20.493331 29694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:20.493676 29694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:20.496939 29694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "237bb398b1f04ca3be1eae10adc19133"
format_stamp: "Formatted at 2026-08-12 07:01:20 on dist-test-slave-t3q3"
server_key: "b1527f86a95683bf8d6c31247631e2a8"
server_key_iv: "d322563ba4efd5f93e70068c65c051c8"
server_key_version: "encryptionkey@0"
I20260812 07:01:20.497779 29694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "237bb398b1f04ca3be1eae10adc19133"
format_stamp: "Formatted at 2026-08-12 07:01:20 on dist-test-slave-t3q3"
server_key: "b1527f86a95683bf8d6c31247631e2a8"
server_key_iv: "d322563ba4efd5f93e70068c65c051c8"
server_key_version: "encryptionkey@0"
I20260812 07:01:20.503446 29694 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 07:01:20.507236 29712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:20.508848 29694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:20.509068 29694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "237bb398b1f04ca3be1eae10adc19133"
format_stamp: "Formatted at 2026-08-12 07:01:20 on dist-test-slave-t3q3"
server_key: "b1527f86a95683bf8d6c31247631e2a8"
server_key_iv: "d322563ba4efd5f93e70068c65c051c8"
server_key_version: "encryptionkey@0"
I20260812 07:01:20.509404 29694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:20.515882 29694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:20.520123 29694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:20.520452 29694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:20.539852 29694 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37781
I20260812 07:01:20.539901 29789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37781 every 8 connection(s)
I20260812 07:01:20.541386 29694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:20.545222 29790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:20.549790 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 29694
I20260812 07:01:20.549988 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:20.550578 27528 external_mini_cluster.cc:1428] Setting key 9b7855ac837ca995a7461b0e5c1bc882
I20260812 07:01:20.557267 29790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133: Bootstrap starting.
I20260812 07:01:20.559122 29795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:20.552497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34925 (local address 127.0.0.1:37781)
0812 07:01:20.553335 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:20.553347 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:20.553388 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:01:20.555668 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:20.555691 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:20.556019 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:20.556418 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:20.557566 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:20.557573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:20.557585 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:20.557600 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:20.557662 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:20.557683 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:20.557866 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:20.557985 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 07:01:20.558184 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":395,"threads_started":1}
I20260812 07:01:20.561019 29790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:20.562525 29790 log.cc:824] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:20.563329 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:20.565872 29790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133: No bootstrap required, opened a new log
I20260812 07:01:20.571166 29795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:20.568656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45635 (local address 127.0.0.1:37781)
0812 07:01:20.568810 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:20.568814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:20.569205 (+ 391us) server_negotiation.cc:407] Connection header received
0812 07:01:20.569383 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:20.569386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:20.569430 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:20.569510 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:20.570487 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:20.570491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:20.570494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:20.570508 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:20.570547 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:20.570563 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:20.570656 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:20.570806 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 07:01:20.570850 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 07:01:20.575582 29790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237bb398b1f04ca3be1eae10adc19133" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } }
I20260812 07:01:20.576143 29790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:20.576272 29790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 237bb398b1f04ca3be1eae10adc19133, State: Initialized, Role: FOLLOWER
I20260812 07:01:20.576910 29790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237bb398b1f04ca3be1eae10adc19133" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } }
I20260812 07:01:20.577168 29790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:20.577329 29790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:20.577514 29790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:20.579733 29790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237bb398b1f04ca3be1eae10adc19133" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } }
I20260812 07:01:20.580358 29790 leader_election.cc:302] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 237bb398b1f04ca3be1eae10adc19133; no voters:
I20260812 07:01:20.580971 29790 leader_election.cc:288] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:20.581112 29806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:20.582899 29806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [term 1 LEADER]: Becoming Leader. State: Replica: 237bb398b1f04ca3be1eae10adc19133, State: Running, Role: LEADER
I20260812 07:01:20.583123 29790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:20.583475 29806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237bb398b1f04ca3be1eae10adc19133" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } }
I20260812 07:01:20.586615 29808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 237bb398b1f04ca3be1eae10adc19133. Latest consensus state: current_term: 1 leader_uuid: "237bb398b1f04ca3be1eae10adc19133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237bb398b1f04ca3be1eae10adc19133" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } } }
I20260812 07:01:20.586587 29807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "237bb398b1f04ca3be1eae10adc19133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237bb398b1f04ca3be1eae10adc19133" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } } }
I20260812 07:01:20.587064 29807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:20.587071 29808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:20.588866 29816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:20.594283 29816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:20.603348 29816 catalog_manager.cc:1347] Generated new cluster ID: 314ea34c4b37494db697c4a0b4943102
I20260812 07:01:20.603484 29816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:20.627856 29816 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:20.628345 29816 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:20.635968 29816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 237bb398b1f04ca3be1eae10adc19133: Generated new TSK 0
I20260812 07:01:20.636988 29816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:20.643906 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29694
W20260812 07:01:20.645077 29785 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34925
W20260812 07:01:20.645846 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29694: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:20.645995 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 29694
2026-08-12T07:01:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:20 dist-test-slave-t3q3 krb5kdc[29833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:20 dist-test-slave-t3q3 krb5kdc[29833](info): set up 2 sockets
Aug 12 07:01:20 dist-test-slave-t3q3 krb5kdc[29833](info): commencing operation
krb5kdc: starting...
W20260812 07:01:22.026111 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.002s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:22 dist-test-slave-t3q3 krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:22.387181 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39453
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:22.750818 29856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:22.751135 29856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:22.751307 29856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:22.758699 29856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:22.758814 29856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:22.758925 29856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:22.759068 29856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:22.759130 29856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:22.769268 29856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:22.770632 29856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:22.771967 29856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:22.784825 29862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:22.784879 29870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:22.784925 29866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:22.785795 29856 server_base.cc:1034] running on GCE node
I20260812 07:01:22.786716 29856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:22.788280 29856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:22.789529 29856 hybrid_clock.cc:648] HybridClock initialized: now 1786518082789463 us; error 52 us; skew 500 ppm
I20260812 07:01:22.789906 29856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:22 dist-test-slave-t3q3 krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518082, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:22.796176 29856 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:22.797773 29856 webserver.cc:460] Webserver started at http://127.0.0.1:32987/ using document root <none> and password file <none>
I20260812 07:01:22.798475 29856 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:22.798629 29856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:22.798954 29856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:22.802048 29856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "01fc123a81f8462f865e2f26cdfb15bc"
format_stamp: "Formatted at 2026-08-12 07:01:22 on dist-test-slave-t3q3"
server_key: "92afa6ae7259a24fd820b24be3df201f"
server_key_iv: "0e2dceb78fad4319c6ee72e915fd7559"
server_key_version: "encryptionkey@0"
I20260812 07:01:22.802839 29856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "01fc123a81f8462f865e2f26cdfb15bc"
format_stamp: "Formatted at 2026-08-12 07:01:22 on dist-test-slave-t3q3"
server_key: "92afa6ae7259a24fd820b24be3df201f"
server_key_iv: "0e2dceb78fad4319c6ee72e915fd7559"
server_key_version: "encryptionkey@0"
I20260812 07:01:22.808168 29856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 07:01:22.811679 29884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:22.813220 29856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:22.813371 29856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "01fc123a81f8462f865e2f26cdfb15bc"
format_stamp: "Formatted at 2026-08-12 07:01:22 on dist-test-slave-t3q3"
server_key: "92afa6ae7259a24fd820b24be3df201f"
server_key_iv: "0e2dceb78fad4319c6ee72e915fd7559"
server_key_version: "encryptionkey@0"
I20260812 07:01:22.813558 29856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:22.839022 29856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:22.845023 29856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:22.845376 29856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:22.865288 29856 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39453
I20260812 07:01:22.865312 29972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39453 every 8 connection(s)
I20260812 07:01:22.866879 29856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:22.870889 29973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:22.877218 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 29856
I20260812 07:01:22.877429 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:22.877867 27528 external_mini_cluster.cc:1428] Setting key b8858c8458738865f20a9861c9f50a35
I20260812 07:01:22.881700 29973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc: Bootstrap starting.
I20260812 07:01:22.885447 29973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:22.886512 29973 log.cc:824] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:22.889618 29973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc: No bootstrap required, opened a new log
Aug 12 07:01:22 dist-test-slave-t3q3 krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:01:22.897372 29973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fc123a81f8462f865e2f26cdfb15bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39453 } }
I20260812 07:01:22.897766 29973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:22.897874 29973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01fc123a81f8462f865e2f26cdfb15bc, State: Initialized, Role: FOLLOWER
I20260812 07:01:22.898599 29973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fc123a81f8462f865e2f26cdfb15bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39453 } }
I20260812 07:01:22.898900 29973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:22.899068 29973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:22.899271 29973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:22.902662 29976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:22.879721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33591 (local address 127.0.0.1:39453)
0812 07:01:22.880554 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:22.880565 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:22.880607 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:01:22.882340 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:22.882363 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:22.882735 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:22.883076 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:22.885094 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:22.886451 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:22.888142 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:22.888799 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:22.888929 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:22.893867 (+ 4938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:22.893876 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:22.893892 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:22.893935 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:22.897360 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:22.898331 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:22.898341 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:22.898349 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:22.898585 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:22.899104 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:22.899109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:22.899114 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:22.899909 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:22.900254 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:22.900771 (+ 517us) server_negotiation.cc:299] Negotiation successful
0812 07:01:22.901265 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":422,"threads_started":1}
I20260812 07:01:22.902812 29973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fc123a81f8462f865e2f26cdfb15bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39453 } }
I20260812 07:01:22.903370 29973 leader_election.cc:302] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01fc123a81f8462f865e2f26cdfb15bc; no voters:
I20260812 07:01:22.904006 29973 leader_election.cc:288] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:22.904203 29978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:22.905632 29978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [term 1 LEADER]: Becoming Leader. State: Replica: 01fc123a81f8462f865e2f26cdfb15bc, State: Running, Role: LEADER
I20260812 07:01:22.906390 29973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:22.906255 29978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fc123a81f8462f865e2f26cdfb15bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39453 } }
I20260812 07:01:22.907445 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:22.917037 29979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01fc123a81f8462f865e2f26cdfb15bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fc123a81f8462f865e2f26cdfb15bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39453 } } }
I20260812 07:01:22.917590 29979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:22.921332 29981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01fc123a81f8462f865e2f26cdfb15bc. Latest consensus state: current_term: 1 leader_uuid: "01fc123a81f8462f865e2f26cdfb15bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fc123a81f8462f865e2f26cdfb15bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39453 } } }
I20260812 07:01:22.921839 29981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:22.922688 29993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:22.929947 29993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:22.943130 29976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:22.924270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54035 (local address 127.0.0.1:39453)
0812 07:01:22.924488 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:22.924495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:22.925625 (+ 1130us) server_negotiation.cc:407] Connection header received
0812 07:01:22.925951 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:22.925957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:22.926056 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:22.926151 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:22.928405 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:22.929535 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:22.931468 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:22.932197 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:22.932306 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:22.935724 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:22.935731 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:22.935736 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:22.935782 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:22.939749 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:22.940712 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:22.940718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:22.940722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:22.940872 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:22.941373 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:22.941380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:22.941383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:22.941864 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:22.942358 (+ 494us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:22.942663 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 07:01:22.942855 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:01:22.946712 29993 catalog_manager.cc:1347] Generated new cluster ID: a89fa3b91e2d486f957b948e3439fc19
I20260812 07:01:22.946892 29993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:22.957538 29993 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:22.958855 29993 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:22.978633 29993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01fc123a81f8462f865e2f26cdfb15bc: Generated new TSK 0
I20260812 07:01:22.979467 29993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:22.992543 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 29856
I20260812 07:01:22.993923 29968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:23.088426 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 29856
2026-08-12T07:01:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:23 dist-test-slave-t3q3 krb5kdc[30024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:23 dist-test-slave-t3q3 krb5kdc[30024](info): set up 2 sockets
Aug 12 07:01:23 dist-test-slave-t3q3 krb5kdc[30024](info): commencing operation
krb5kdc: starting...
W20260812 07:01:24.491485 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.357s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:24 dist-test-slave-t3q3 krb5kdc[30024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:24.853302 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:25.209585 30045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:25.209949 30045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:25.210135 30045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:25.217576 30045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:25.217742 30045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:25.217841 30045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:25.217958 30045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:25.218045 30045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:25.228132 30045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:25.229538 30045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:25.230868 30045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:25.243151 30052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:25.243047 30057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:25.243104 30053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:25.243918 30045 server_base.cc:1034] running on GCE node
I20260812 07:01:25.244722 30045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:25.246172 30045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:25.247399 30045 hybrid_clock.cc:648] HybridClock initialized: now 1786518085247346 us; error 48 us; skew 500 ppm
I20260812 07:01:25.247777 30045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:25 dist-test-slave-t3q3 krb5kdc[30024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518085, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:25.251142 30045 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:25.252749 30045 webserver.cc:460] Webserver started at http://127.0.0.1:38267/ using document root <none> and password file <none>
I20260812 07:01:25.253443 30045 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:25.253597 30045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:25.253909 30045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:25.256958 30045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "1ff6931d086d43ce87e71bba66eaea77"
format_stamp: "Formatted at 2026-08-12 07:01:25 on dist-test-slave-t3q3"
server_key: "326d9414fb6c316d778a2eac1d709ce1"
server_key_iv: "edc1e32c24b8f752c88f0e47f28787b9"
server_key_version: "encryptionkey@0"
I20260812 07:01:25.257759 30045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "1ff6931d086d43ce87e71bba66eaea77"
format_stamp: "Formatted at 2026-08-12 07:01:25 on dist-test-slave-t3q3"
server_key: "326d9414fb6c316d778a2eac1d709ce1"
server_key_iv: "edc1e32c24b8f752c88f0e47f28787b9"
server_key_version: "encryptionkey@0"
I20260812 07:01:25.263024 30045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:01:25.266454 30065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:25.267696 30045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:01:25.267885 30045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "1ff6931d086d43ce87e71bba66eaea77"
format_stamp: "Formatted at 2026-08-12 07:01:25 on dist-test-slave-t3q3"
server_key: "326d9414fb6c316d778a2eac1d709ce1"
server_key_iv: "edc1e32c24b8f752c88f0e47f28787b9"
server_key_version: "encryptionkey@0"
I20260812 07:01:25.268110 30045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:25.299739 30045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:25.304001 30045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:25.304334 30045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:25.323645 30045 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33593
I20260812 07:01:25.323696 30150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33593 every 8 connection(s)
I20260812 07:01:25.325093 30045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:25.329095 30151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:25.330253 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 30045
I20260812 07:01:25.330449 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:25.331044 27528 external_mini_cluster.cc:1428] Setting key 1847be3ed1461b475da00486375ab6cb
I20260812 07:01:25.342890 30151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77: Bootstrap starting.
I20260812 07:01:25.345288 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:25.333037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42441 (local address 127.0.0.1:33593)
0812 07:01:25.333918 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:25.333931 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:25.333972 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:01:25.336194 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:25.336219 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:25.336628 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:25.337041 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:25.337582 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:25.338871 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:25.340688 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:25.341404 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:25.341612 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:25.342880 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:25.342888 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:25.342903 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:25.343508 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:25.343593 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:25.343619 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:25.343754 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:25.343997 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 07:01:25.344299 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":366,"threads_started":1}
I20260812 07:01:25.346642 30151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:25.347860 30151 log.cc:824] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:25.349346 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:25.350826 30151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77: No bootstrap required, opened a new log
I20260812 07:01:25.360435 30151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff6931d086d43ce87e71bba66eaea77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } }
I20260812 07:01:25.360862 30151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:25.361001 30151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ff6931d086d43ce87e71bba66eaea77, State: Initialized, Role: FOLLOWER
I20260812 07:01:25.361615 30151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff6931d086d43ce87e71bba66eaea77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } }
I20260812 07:01:25.361889 30151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:25.362054 30151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:25.362324 30151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:25.362427 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:25.353837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40207 (local address 127.0.0.1:33593)
0812 07:01:25.353999 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:25.354004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:25.356622 (+ 2618us) server_negotiation.cc:407] Connection header received
0812 07:01:25.356862 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:25.356866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:25.356933 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:25.357051 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:25.357572 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:25.358335 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:25.359690 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:25.360205 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:25.360343 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:25.361584 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:25.361589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:25.361593 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:25.361614 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:25.361652 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:25.361670 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:25.361777 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:25.361959 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 07:01:25.362093 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:01:25.365268 30151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff6931d086d43ce87e71bba66eaea77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } }
I20260812 07:01:25.365942 30151 leader_election.cc:302] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ff6931d086d43ce87e71bba66eaea77; no voters:
I20260812 07:01:25.366593 30151 leader_election.cc:288] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:25.366688 30165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:25.367694 30165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [term 1 LEADER]: Becoming Leader. State: Replica: 1ff6931d086d43ce87e71bba66eaea77, State: Running, Role: LEADER
I20260812 07:01:25.368383 30165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff6931d086d43ce87e71bba66eaea77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } }
I20260812 07:01:25.369180 30151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:25.371232 30166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ff6931d086d43ce87e71bba66eaea77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff6931d086d43ce87e71bba66eaea77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } } }
I20260812 07:01:25.371618 30166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:25.372471 30167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ff6931d086d43ce87e71bba66eaea77. Latest consensus state: current_term: 1 leader_uuid: "1ff6931d086d43ce87e71bba66eaea77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff6931d086d43ce87e71bba66eaea77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } } }
I20260812 07:01:25.372763 30167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:25.374516 30171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:25.379961 30171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:25.389761 30171 catalog_manager.cc:1347] Generated new cluster ID: dbb5a0f5274d4722a5a8d718f3cb274e
I20260812 07:01:25.389909 30171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:25.408144 30171 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:25.409543 30171 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:25.425406 30171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ff6931d086d43ce87e71bba66eaea77: Generated new TSK 0
I20260812 07:01:25.426355 30171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:25.445647 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30045
W20260812 07:01:25.446884 30145 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42441
W20260812 07:01:25.447697 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30045: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:25.447855 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 30045
2026-08-12T07:01:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2358 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:25 dist-test-slave-t3q3 krb5kdc[30194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:25 dist-test-slave-t3q3 krb5kdc[30194](info): set up 2 sockets
Aug 12 07:01:25 dist-test-slave-t3q3 krb5kdc[30194](info): commencing operation
krb5kdc: starting...
W20260812 07:01:26.846100 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:27 dist-test-slave-t3q3 krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:27.207404 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33151
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:27.588999 30216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:27.589365 30216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:27.589534 30216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:27.597126 30216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:27.597304 30216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:27.597417 30216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:27.597534 30216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:27.597613 30216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:27.608080 30216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:27.609447 30216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:27.610829 30216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:27.623486 30227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:27.623499 30228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:27.623811 30232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:27.624317 30216 server_base.cc:1034] running on GCE node
I20260812 07:01:27.625247 30216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:27.626814 30216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:27.628047 30216 hybrid_clock.cc:648] HybridClock initialized: now 1786518087627977 us; error 54 us; skew 500 ppm
I20260812 07:01:27.628500 30216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:27 dist-test-slave-t3q3 krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518087, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:27.631892 30216 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:27.633415 30216 webserver.cc:460] Webserver started at http://127.0.0.1:46143/ using document root <none> and password file <none>
I20260812 07:01:27.634061 30216 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:27.634168 30216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:27.634431 30216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:27.637482 30216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "15bc12fffd924a4f95b62a186cd38208"
format_stamp: "Formatted at 2026-08-12 07:01:27 on dist-test-slave-t3q3"
server_key: "bf080071b99e38063a506fd35bffad7d"
server_key_iv: "0212ec925f5a546fadbce1ec753e0c92"
server_key_version: "encryptionkey@0"
I20260812 07:01:27.638224 30216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "15bc12fffd924a4f95b62a186cd38208"
format_stamp: "Formatted at 2026-08-12 07:01:27 on dist-test-slave-t3q3"
server_key: "bf080071b99e38063a506fd35bffad7d"
server_key_iv: "0212ec925f5a546fadbce1ec753e0c92"
server_key_version: "encryptionkey@0"
I20260812 07:01:27.643525 30216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 07:01:27.647185 30244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:27.648612 30216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:27.648805 30216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "15bc12fffd924a4f95b62a186cd38208"
format_stamp: "Formatted at 2026-08-12 07:01:27 on dist-test-slave-t3q3"
server_key: "bf080071b99e38063a506fd35bffad7d"
server_key_iv: "0212ec925f5a546fadbce1ec753e0c92"
server_key_version: "encryptionkey@0"
I20260812 07:01:27.648988 30216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:27.654788 30216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:27.659053 30216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:27.659366 30216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:27.679260 30216 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33151
I20260812 07:01:27.679435 30333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33151 every 8 connection(s)
I20260812 07:01:27.680893 30216 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:27.685053 30335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:27.687966 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 30216
I20260812 07:01:27.688346 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:27.688855 27528 external_mini_cluster.cc:1428] Setting key 95222a5b93b4122c107a45f971d58757
I20260812 07:01:27.697835 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:27.690889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57433 (local address 127.0.0.1:33151)
0812 07:01:27.691848 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:27.691861 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:27.691909 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:01:27.693972 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:27.693998 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:27.694391 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:27.694773 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:27.696053 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:27.696115 (+ 62us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:27.696137 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:27.696156 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:27.696228 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:27.696258 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:27.696452 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:27.696583 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 07:01:27.696816 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":440,"threads_started":1}
I20260812 07:01:27.698318 30335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208: Bootstrap starting.
I20260812 07:01:27.701699 30335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:27.701924 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:27.702826 30335 log.cc:824] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:27.706410 30335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208: No bootstrap required, opened a new log
I20260812 07:01:27.709136 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:27.706565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56761 (local address 127.0.0.1:33151)
0812 07:01:27.706750 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:27.706756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:27.707149 (+ 393us) server_negotiation.cc:407] Connection header received
0812 07:01:27.707286 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:27.707293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:27.707351 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:27.707460 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:27.708459 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:27.708477 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:27.708483 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:27.708502 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:27.708562 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:27.708582 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:27.708710 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:27.708780 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 07:01:27.708846 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:01:27.714728 30335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bc12fffd924a4f95b62a186cd38208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33151 } }
I20260812 07:01:27.715087 30335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:27.715163 30335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15bc12fffd924a4f95b62a186cd38208, State: Initialized, Role: FOLLOWER
I20260812 07:01:27.715672 30335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bc12fffd924a4f95b62a186cd38208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33151 } }
I20260812 07:01:27.715880 30335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:27.715986 30335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:27.716177 30335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:27.718397 30335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bc12fffd924a4f95b62a186cd38208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33151 } }
I20260812 07:01:27.718845 30335 leader_election.cc:302] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15bc12fffd924a4f95b62a186cd38208; no voters:
I20260812 07:01:27.719429 30335 leader_election.cc:288] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:27.719600 30351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:27.720323 30351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [term 1 LEADER]: Becoming Leader. State: Replica: 15bc12fffd924a4f95b62a186cd38208, State: Running, Role: LEADER
I20260812 07:01:27.721036 30351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bc12fffd924a4f95b62a186cd38208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33151 } }
I20260812 07:01:27.721733 30335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:27.724380 30354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15bc12fffd924a4f95b62a186cd38208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bc12fffd924a4f95b62a186cd38208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33151 } } }
I20260812 07:01:27.724303 30355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15bc12fffd924a4f95b62a186cd38208. Latest consensus state: current_term: 1 leader_uuid: "15bc12fffd924a4f95b62a186cd38208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bc12fffd924a4f95b62a186cd38208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33151 } } }
I20260812 07:01:27.724941 30354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:27.724975 30355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:27.729548 30364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:27.734069 30364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:27.743032 30364 catalog_manager.cc:1347] Generated new cluster ID: 277eb9cc74d3416cafe391174062b159
I20260812 07:01:27.743175 30364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:27.772444 30364 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:27.772929 30364 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:27.782385 30364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15bc12fffd924a4f95b62a186cd38208: Generated new TSK 0
I20260812 07:01:27.783291 30364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:27.805114 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30216
W20260812 07:01:27.806353 30325 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57433
W20260812 07:01:27.807096 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30216: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:27.807267 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 30216
2026-08-12T07:01:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:27 dist-test-slave-t3q3 krb5kdc[30382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:27 dist-test-slave-t3q3 krb5kdc[30382](info): set up 2 sockets
Aug 12 07:01:27 dist-test-slave-t3q3 krb5kdc[30382](info): commencing operation
krb5kdc: starting...
W20260812 07:01:29.204849 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.002s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:29 dist-test-slave-t3q3 krb5kdc[30382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:29.559254 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:29.912686 30405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:29.913034 30405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:29.913259 30405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:29.920629 30405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:29.920761 30405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:29.920851 30405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:29.920961 30405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:29.921056 30405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:29.930984 30405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:29.932360 30405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:29.933691 30405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:29.945672 30418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:29.945715 30415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:29.945763 30420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:29.952243 30405 server_base.cc:1034] running on GCE node
I20260812 07:01:29.953178 30405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:29.954715 30405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:29.955963 30405 hybrid_clock.cc:648] HybridClock initialized: now 1786518089955918 us; error 69 us; skew 500 ppm
I20260812 07:01:29.956395 30405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:29 dist-test-slave-t3q3 krb5kdc[30382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518089, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:29.959637 30405 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:29.961162 30405 webserver.cc:460] Webserver started at http://127.0.0.1:46319/ using document root <none> and password file <none>
I20260812 07:01:29.961869 30405 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:29.961978 30405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:29.962242 30405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:29.965410 30405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "eb07a032795748758dded5f5115b520d"
format_stamp: "Formatted at 2026-08-12 07:01:29 on dist-test-slave-t3q3"
server_key: "37f86c44076b04cb247ca1f849c19247"
server_key_iv: "3d2f2b5b1c1cfa16715698c01fb4f056"
server_key_version: "encryptionkey@0"
I20260812 07:01:29.966187 30405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "eb07a032795748758dded5f5115b520d"
format_stamp: "Formatted at 2026-08-12 07:01:29 on dist-test-slave-t3q3"
server_key: "37f86c44076b04cb247ca1f849c19247"
server_key_iv: "3d2f2b5b1c1cfa16715698c01fb4f056"
server_key_version: "encryptionkey@0"
I20260812 07:01:29.971491 30405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260812 07:01:29.975238 30432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:29.976734 30405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:29.976935 30405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "eb07a032795748758dded5f5115b520d"
format_stamp: "Formatted at 2026-08-12 07:01:29 on dist-test-slave-t3q3"
server_key: "37f86c44076b04cb247ca1f849c19247"
server_key_iv: "3d2f2b5b1c1cfa16715698c01fb4f056"
server_key_version: "encryptionkey@0"
I20260812 07:01:29.977121 30405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:29.993067 30405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:29.997896 30405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:29.998212 30405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:30.018167 30405 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37609
I20260812 07:01:30.018191 30509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37609 every 8 connection(s)
I20260812 07:01:30.019757 30405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:30.023748 30514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:30.024502 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 30405
I20260812 07:01:30.024686 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:30.025127 27528 external_mini_cluster.cc:1428] Setting key 1dd2466e2d412ee10e568bd263ebb86d
I20260812 07:01:30.036414 30514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d: Bootstrap starting.
Aug 12 07:01:30 dist-test-slave-t3q3 krb5kdc[30382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:01:30.039506 30514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:30.040622 30514 log.cc:824] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:30.043520 30514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d: No bootstrap required, opened a new log
I20260812 07:01:30.048633 30517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:30.027104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:34009 (local address 127.0.0.1:37609)
0812 07:01:30.027917 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:30.027929 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:30.027969 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:01:30.030135 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:30.030161 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:30.030569 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:30.030965 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:30.032567 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:30.033915 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:30.035253 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:30.035924 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:30.036164 (+ 240us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:30.040618 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:30.040626 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:30.040641 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:30.040679 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:30.044138 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:30.044958 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:30.044969 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:30.044976 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:30.045160 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:30.045670 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:30.045674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:30.045676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:30.046239 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:30.046607 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:30.047154 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 07:01:30.047474 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":388,"threads_started":1}
I20260812 07:01:30.051757 30514 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb07a032795748758dded5f5115b520d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37609 } }
I20260812 07:01:30.052174 30514 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:30.052307 30514 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb07a032795748758dded5f5115b520d, State: Initialized, Role: FOLLOWER
I20260812 07:01:30.052731 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:30.052960 30514 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb07a032795748758dded5f5115b520d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37609 } }
I20260812 07:01:30.053232 30514 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:30.053386 30514 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:30.053570 30514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:30.056030 30514 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb07a032795748758dded5f5115b520d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37609 } }
I20260812 07:01:30.056589 30514 leader_election.cc:302] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb07a032795748758dded5f5115b520d; no voters:
I20260812 07:01:30.057322 30514 leader_election.cc:288] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:30.057471 30531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:30.059621 30514 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:30.060200 30531 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [term 1 LEADER]: Becoming Leader. State: Replica: eb07a032795748758dded5f5115b520d, State: Running, Role: LEADER
I20260812 07:01:30.060845 30531 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb07a032795748758dded5f5115b520d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37609 } }
I20260812 07:01:30.064345 30533 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb07a032795748758dded5f5115b520d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb07a032795748758dded5f5115b520d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37609 } } }
I20260812 07:01:30.064288 30534 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb07a032795748758dded5f5115b520d. Latest consensus state: current_term: 1 leader_uuid: "eb07a032795748758dded5f5115b520d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb07a032795748758dded5f5115b520d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37609 } } }
I20260812 07:01:30.064975 30534 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:30.065176 30533 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:30.078512 30547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:30.080536 30517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:30.057479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:44819 (local address 127.0.0.1:37609)
0812 07:01:30.057681 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:30.057686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:30.058338 (+ 652us) server_negotiation.cc:407] Connection header received
0812 07:01:30.058594 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:30.058600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:30.058693 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:30.058756 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:30.060647 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:30.061926 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:30.063862 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:30.065222 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:30.065528 (+ 306us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:30.070286 (+ 4758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:30.070297 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:30.070305 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:30.070362 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:30.075493 (+ 5131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:30.077081 (+ 1588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:30.077087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:30.077092 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:30.077332 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:30.078200 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:30.078209 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:30.078214 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:30.078858 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:30.079234 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:30.079748 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 07:01:30.080013 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:01:30.084653 30547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:30.094662 30547 catalog_manager.cc:1347] Generated new cluster ID: 0c83aab523d34c658e3d5951e4cdcd1e
I20260812 07:01:30.094938 30547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:30.114580 30547 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:30.116521 30547 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:30.123908 30547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb07a032795748758dded5f5115b520d: Generated new TSK 0
I20260812 07:01:30.124953 30547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:30.139284 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30405
I20260812 07:01:30.140879 30503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:30.233615 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 30405
W20260812 07:01:30.261041 30379 connection.cc:535] client connection to 127.0.0.1:37609 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37609): Connection reset by peer (error 104)
2026-08-12T07:01:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:30 dist-test-slave-t3q3 krb5kdc[30571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:30 dist-test-slave-t3q3 krb5kdc[30571](info): set up 2 sockets
Aug 12 07:01:30 dist-test-slave-t3q3 krb5kdc[30571](info): commencing operation
krb5kdc: starting...
W20260812 07:01:31.625177 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.007s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:31 dist-test-slave-t3q3 krb5kdc[30571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:31.996686 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:32.360688 30592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:32.361035 30592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:32.361220 30592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:32.368774 30592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:32.368903 30592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:32.369004 30592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:32.369133 30592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:32.369199 30592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:32.380227 30592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:32.381856 30592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:32.383683 30592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:32.396924 30598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:32.396975 30599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:32.397058 30603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:32.398321 30592 server_base.cc:1034] running on GCE node
I20260812 07:01:32.399250 30592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:32.400969 30592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:32.402267 30592 hybrid_clock.cc:648] HybridClock initialized: now 1786518092402207 us; error 70 us; skew 500 ppm
I20260812 07:01:32.402705 30592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:32 dist-test-slave-t3q3 krb5kdc[30571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518092, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:32.406088 30592 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:32.407720 30592 webserver.cc:460] Webserver started at http://127.0.0.1:43463/ using document root <none> and password file <none>
I20260812 07:01:32.408499 30592 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:32.408627 30592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:32.408896 30592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:32.411962 30592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "f8d1b3f8ca4e4d928dbde839550766f4"
format_stamp: "Formatted at 2026-08-12 07:01:32 on dist-test-slave-t3q3"
server_key: "6ef74eba3a7dbc8377493efd3255634e"
server_key_iv: "c20a2719d64fc2332432cf1f7d818d98"
server_key_version: "encryptionkey@0"
I20260812 07:01:32.412825 30592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "f8d1b3f8ca4e4d928dbde839550766f4"
format_stamp: "Formatted at 2026-08-12 07:01:32 on dist-test-slave-t3q3"
server_key: "6ef74eba3a7dbc8377493efd3255634e"
server_key_iv: "c20a2719d64fc2332432cf1f7d818d98"
server_key_version: "encryptionkey@0"
I20260812 07:01:32.418212 30592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:01:32.422049 30610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:32.423519 30592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:01:32.423691 30592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "f8d1b3f8ca4e4d928dbde839550766f4"
format_stamp: "Formatted at 2026-08-12 07:01:32 on dist-test-slave-t3q3"
server_key: "6ef74eba3a7dbc8377493efd3255634e"
server_key_iv: "c20a2719d64fc2332432cf1f7d818d98"
server_key_version: "encryptionkey@0"
I20260812 07:01:32.423866 30592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:32.443095 30592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:32.447978 30592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:32.448325 30592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:32.467906 30592 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41335
I20260812 07:01:32.467931 30701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41335 every 8 connection(s)
I20260812 07:01:32.469375 30592 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:32.473320 30702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:32.473703 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 30592
I20260812 07:01:32.473896 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:32.474488 27528 external_mini_cluster.cc:1428] Setting key 44dd6490105796a95d6314d7187f4964
I20260812 07:01:32.487663 30702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4: Bootstrap starting.
I20260812 07:01:32.489274 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:32.476478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:57469 (local address 127.0.0.1:41335)
0812 07:01:32.477329 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:32.477340 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:32.477382 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:01:32.479659 (+ 2277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:32.479688 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:32.480190 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:32.480561 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:32.481155 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:32.482404 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:32.484506 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:32.485259 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:32.485408 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:32.486807 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:32.486815 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:32.486831 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:32.487390 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:32.487469 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:32.487513 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:32.487642 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:32.488057 (+ 415us) server_negotiation.cc:299] Negotiation successful
0812 07:01:32.488293 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":397,"threads_started":1}
I20260812 07:01:32.491345 30702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:32.492683 30702 log.cc:824] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:32.493628 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:32.496265 30702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4: No bootstrap required, opened a new log
I20260812 07:01:32.506124 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:32.498833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:43873 (local address 127.0.0.1:41335)
0812 07:01:32.499029 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:32.499035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:32.499548 (+ 513us) server_negotiation.cc:407] Connection header received
0812 07:01:32.499718 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:32.499724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:32.499811 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:32.499927 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:32.500505 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:32.501722 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:32.503151 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:32.503870 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:32.504018 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:32.505242 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:32.505249 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:32.505254 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:32.505276 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:32.505324 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:32.505344 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:32.505465 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:32.505726 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 07:01:32.505787 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:01:32.506367 30702 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d1b3f8ca4e4d928dbde839550766f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41335 } }
I20260812 07:01:32.506855 30702 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:32.506987 30702 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8d1b3f8ca4e4d928dbde839550766f4, State: Initialized, Role: FOLLOWER
I20260812 07:01:32.507634 30702 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d1b3f8ca4e4d928dbde839550766f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41335 } }
I20260812 07:01:32.507856 30702 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:32.507961 30702 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:32.508172 30702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:32.510366 30702 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d1b3f8ca4e4d928dbde839550766f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41335 } }
I20260812 07:01:32.510821 30702 leader_election.cc:302] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8d1b3f8ca4e4d928dbde839550766f4; no voters:
I20260812 07:01:32.511349 30702 leader_election.cc:288] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:32.511509 30715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:32.512656 30715 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [term 1 LEADER]: Becoming Leader. State: Replica: f8d1b3f8ca4e4d928dbde839550766f4, State: Running, Role: LEADER
I20260812 07:01:32.513386 30715 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d1b3f8ca4e4d928dbde839550766f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41335 } }
I20260812 07:01:32.513835 30702 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:32.516237 30718 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8d1b3f8ca4e4d928dbde839550766f4. Latest consensus state: current_term: 1 leader_uuid: "f8d1b3f8ca4e4d928dbde839550766f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d1b3f8ca4e4d928dbde839550766f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41335 } } }
I20260812 07:01:32.516677 30718 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:32.517267 30716 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8d1b3f8ca4e4d928dbde839550766f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d1b3f8ca4e4d928dbde839550766f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41335 } } }
I20260812 07:01:32.518066 30716 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:32.523638 30729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:32.529001 30729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:32.539582 30729 catalog_manager.cc:1347] Generated new cluster ID: 81ece527a9f3485ba516e91b3ae91b23
I20260812 07:01:32.539723 30729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:32.556003 30729 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:32.557448 30729 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:32.571141 30729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8d1b3f8ca4e4d928dbde839550766f4: Generated new TSK 0
I20260812 07:01:32.572247 30729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:32.578469 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30592
W20260812 07:01:32.579797 30696 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.50:57469
W20260812 07:01:32.580785 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30592: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:32.580973 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 30592
W20260812 07:01:32.604280 30569 connection.cc:535] client connection to 127.0.0.1:41335 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41335): Connection reset by peer (error 104)
2026-08-12T07:01:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:32 dist-test-slave-t3q3 krb5kdc[30752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:32 dist-test-slave-t3q3 krb5kdc[30752](info): set up 2 sockets
Aug 12 07:01:32 dist-test-slave-t3q3 krb5kdc[30752](info): commencing operation
krb5kdc: starting...
W20260812 07:01:33.981381 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.357s user 0.004s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:34 dist-test-slave-t3q3 krb5kdc[30752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:34.331625 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:34.691574 30768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:34.691865 30768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:34.691960 30768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:34.699255 30768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:34.699365 30768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:34.699414 30768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:34.699476 30768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:34.699517 30768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:34.709328 30768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:34.710578 30768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:34.711844 30768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:34.724108 30779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:34.724290 30784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:34.724195 30781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:34.725268 30768 server_base.cc:1034] running on GCE node
I20260812 07:01:34.726123 30768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:34.727593 30768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:34.728871 30768 hybrid_clock.cc:648] HybridClock initialized: now 1786518094728807 us; error 40 us; skew 500 ppm
I20260812 07:01:34.729394 30768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:34 dist-test-slave-t3q3 krb5kdc[30752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518094, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:34.738014 30768 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:34.740185 30768 webserver.cc:460] Webserver started at http://127.0.0.1:37737/ using document root <none> and password file <none>
I20260812 07:01:34.741240 30768 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:34.741449 30768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:34.741816 30768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:34.745194 30768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "94faa435a9f54049be4d43d6298fdf68"
format_stamp: "Formatted at 2026-08-12 07:01:34 on dist-test-slave-t3q3"
server_key: "c6dad07389ae03ac3f6c7b7f56f7384b"
server_key_iv: "2721b33e493287406fdf31fdaa78a801"
server_key_version: "encryptionkey@0"
I20260812 07:01:34.746045 30768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "94faa435a9f54049be4d43d6298fdf68"
format_stamp: "Formatted at 2026-08-12 07:01:34 on dist-test-slave-t3q3"
server_key: "c6dad07389ae03ac3f6c7b7f56f7384b"
server_key_iv: "2721b33e493287406fdf31fdaa78a801"
server_key_version: "encryptionkey@0"
I20260812 07:01:34.751394 30768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:01:34.755183 30796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:34.756729 30768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:34.756917 30768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "94faa435a9f54049be4d43d6298fdf68"
format_stamp: "Formatted at 2026-08-12 07:01:34 on dist-test-slave-t3q3"
server_key: "c6dad07389ae03ac3f6c7b7f56f7384b"
server_key_iv: "2721b33e493287406fdf31fdaa78a801"
server_key_version: "encryptionkey@0"
I20260812 07:01:34.757122 30768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:34.763000 30768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:34.767134 30768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:34.767437 30768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:34.786015 30768 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44935
I20260812 07:01:34.786072 30881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44935 every 8 connection(s)
I20260812 07:01:34.787472 30768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:34.791054 30882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:34.796732 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 30768
I20260812 07:01:34.796932 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:34.797408 27528 external_mini_cluster.cc:1428] Setting key ecf0fa59a3842986154651557cdd1261
I20260812 07:01:34.802896 30882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68: Bootstrap starting.
I20260812 07:01:34.806800 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:34.799459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:48903 (local address 127.0.0.1:44935)
0812 07:01:34.800411 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:34.800428 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:34.800482 (+ 54us) server_negotiation.cc:407] Connection header received
0812 07:01:34.802880 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:34.802913 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:34.803361 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:34.803792 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:34.804839 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:34.804850 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:34.804869 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:34.804895 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:34.804967 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:34.804995 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:34.805251 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:34.805401 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 07:01:34.805594 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":395,"threads_started":1}
I20260812 07:01:34.807132 30882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:34.808326 30882 log.cc:824] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:34.810803 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:34.811447 30882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68: No bootstrap required, opened a new log
I20260812 07:01:34.817883 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:34.815432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:58447 (local address 127.0.0.1:44935)
0812 07:01:34.815621 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:34.815629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:34.815972 (+ 343us) server_negotiation.cc:407] Connection header received
0812 07:01:34.816120 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:34.816125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:34.816185 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:34.816306 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:34.817207 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:34.817213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:34.817218 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:34.817237 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:34.817293 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:34.817312 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:34.817438 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:34.817508 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 07:01:34.817572 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:01:34.820698 30882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94faa435a9f54049be4d43d6298fdf68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } }
I20260812 07:01:34.821064 30882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:34.821151 30882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94faa435a9f54049be4d43d6298fdf68, State: Initialized, Role: FOLLOWER
I20260812 07:01:34.821671 30882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94faa435a9f54049be4d43d6298fdf68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } }
I20260812 07:01:34.821895 30882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:34.821995 30882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:34.822137 30882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:34.824628 30882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94faa435a9f54049be4d43d6298fdf68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } }
I20260812 07:01:34.825192 30882 leader_election.cc:302] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94faa435a9f54049be4d43d6298fdf68; no voters:
I20260812 07:01:34.825989 30882 leader_election.cc:288] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:34.826143 30896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:34.826946 30896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [term 1 LEADER]: Becoming Leader. State: Replica: 94faa435a9f54049be4d43d6298fdf68, State: Running, Role: LEADER
I20260812 07:01:34.827572 30896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94faa435a9f54049be4d43d6298fdf68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } }
I20260812 07:01:34.828450 30882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:34.830874 30898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94faa435a9f54049be4d43d6298fdf68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94faa435a9f54049be4d43d6298fdf68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } } }
I20260812 07:01:34.831470 30898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:34.830885 30901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94faa435a9f54049be4d43d6298fdf68. Latest consensus state: current_term: 1 leader_uuid: "94faa435a9f54049be4d43d6298fdf68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94faa435a9f54049be4d43d6298fdf68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } } }
I20260812 07:01:34.836813 30901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:34.837657 30906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:34.844462 30906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:34.854470 30906 catalog_manager.cc:1347] Generated new cluster ID: 2f8f71fa41514f738e0f79b5d1575828
I20260812 07:01:34.854612 30906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:34.889580 30906 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:34.890064 30906 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:34.896504 30906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94faa435a9f54049be4d43d6298fdf68: Generated new TSK 0
I20260812 07:01:34.897395 30906 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:34.904366 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30768
W20260812 07:01:34.905575 30876 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.50:48903
W20260812 07:01:34.906350 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30768: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:34.906503 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 30768
2026-08-12T07:01:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:35 dist-test-slave-t3q3 krb5kdc[30932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:35 dist-test-slave-t3q3 krb5kdc[30932](info): set up 2 sockets
Aug 12 07:01:35 dist-test-slave-t3q3 krb5kdc[30932](info): commencing operation
krb5kdc: starting...
W20260812 07:01:36.291808 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:36 dist-test-slave-t3q3 krb5kdc[30932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:36.653507 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38053
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:37.013365 30951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:37.013731 30951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:37.013909 30951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:37.021078 30951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:37.021241 30951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:37.021370 30951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:37.021477 30951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:37.021565 30951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:37.031177 30951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38053
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:37.032569 30951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:37.033924 30951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:37.046409 30958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:37.046353 30962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:37.046371 30959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:37.047260 30951 server_base.cc:1034] running on GCE node
I20260812 07:01:37.048244 30951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:37.049844 30951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:37.051076 30951 hybrid_clock.cc:648] HybridClock initialized: now 1786518097050998 us; error 77 us; skew 500 ppm
I20260812 07:01:37.051493 30951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:37 dist-test-slave-t3q3 krb5kdc[30932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518097, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:37.054908 30951 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:37.056566 30951 webserver.cc:460] Webserver started at http://127.0.0.1:34517/ using document root <none> and password file <none>
I20260812 07:01:37.057286 30951 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:37.057425 30951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:37.057770 30951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:37.060999 30951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "b21a55a004b44b57ab54ce49daadff96"
format_stamp: "Formatted at 2026-08-12 07:01:37 on dist-test-slave-t3q3"
server_key: "d3ab3b4dfa5e08a70d4fd6453c5b43a2"
server_key_iv: "f8e40731bd4a2ad53bdb26b299b0bf12"
server_key_version: "encryptionkey@0"
I20260812 07:01:37.061815 30951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "b21a55a004b44b57ab54ce49daadff96"
format_stamp: "Formatted at 2026-08-12 07:01:37 on dist-test-slave-t3q3"
server_key: "d3ab3b4dfa5e08a70d4fd6453c5b43a2"
server_key_iv: "f8e40731bd4a2ad53bdb26b299b0bf12"
server_key_version: "encryptionkey@0"
I20260812 07:01:37.067255 30951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:01:37.070986 30970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:37.072481 30951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:37.072695 30951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "b21a55a004b44b57ab54ce49daadff96"
format_stamp: "Formatted at 2026-08-12 07:01:37 on dist-test-slave-t3q3"
server_key: "d3ab3b4dfa5e08a70d4fd6453c5b43a2"
server_key_iv: "f8e40731bd4a2ad53bdb26b299b0bf12"
server_key_version: "encryptionkey@0"
I20260812 07:01:37.072886 30951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:37.106051 30951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:37.110286 30951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:37.110592 30951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:37.129576 30951 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38053
I20260812 07:01:37.129619 31049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38053 every 8 connection(s)
I20260812 07:01:37.130939 30951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:37.134649 31050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:37.137908 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 30951
I20260812 07:01:37.138083 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:37.138640 27528 external_mini_cluster.cc:1428] Setting key f9811167d074228d2765fc6f16716988
I20260812 07:01:37.147138 31050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96: Bootstrap starting.
I20260812 07:01:37.150875 31050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:37.151818 31050 log.cc:824] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:37.154636 31050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96: No bootstrap required, opened a new log
Aug 12 07:01:37 dist-test-slave-t3q3 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:01:37.163259 31050 raft_consensus.cc:348] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21a55a004b44b57ab54ce49daadff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } }
I20260812 07:01:37.163669 31050 raft_consensus.cc:374] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:37.163796 31050 raft_consensus.cc:729] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b21a55a004b44b57ab54ce49daadff96, State: Initialized, Role: FOLLOWER
I20260812 07:01:37.164451 31050 consensus_queue.cc:260] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21a55a004b44b57ab54ce49daadff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } }
I20260812 07:01:37.164726 31050 raft_consensus.cc:388] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:37.164903 31050 raft_consensus.cc:482] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:37.165131 31050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:37.166376 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:37.140530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:51709 (local address 127.0.0.1:38053)
0812 07:01:37.141288 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:37.141298 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:37.141339 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:01:37.142998 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:37.143019 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:37.143409 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:37.143720 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:37.145607 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:37.146899 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:37.148779 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:37.149445 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:37.149571 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:37.158412 (+ 8841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:37.158420 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:37.158438 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:37.158477 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:37.161791 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:37.162627 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:37.162634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:37.162640 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:37.162800 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:37.163289 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:37.163292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:37.163294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:37.163949 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:37.164372 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:37.164981 (+ 609us) server_negotiation.cc:299] Negotiation successful
0812 07:01:37.165369 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":336,"threads_started":1}
I20260812 07:01:37.167634 31050 raft_consensus.cc:504] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21a55a004b44b57ab54ce49daadff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } }
I20260812 07:01:37.168251 31050 leader_election.cc:302] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b21a55a004b44b57ab54ce49daadff96; no voters:
I20260812 07:01:37.168947 31050 leader_election.cc:288] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:37.169062 31062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:37.170360 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:37.170421 31062 raft_consensus.cc:686] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [term 1 LEADER]: Becoming Leader. State: Replica: b21a55a004b44b57ab54ce49daadff96, State: Running, Role: LEADER
I20260812 07:01:37.171057 31062 consensus_queue.cc:237] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21a55a004b44b57ab54ce49daadff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } }
I20260812 07:01:37.171428 31050 sys_catalog.cc:564] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:37.174031 31063 sys_catalog.cc:455] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b21a55a004b44b57ab54ce49daadff96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21a55a004b44b57ab54ce49daadff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } } }
I20260812 07:01:37.174520 31063 sys_catalog.cc:458] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:37.176918 31073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:37.176864 31065 sys_catalog.cc:455] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b21a55a004b44b57ab54ce49daadff96. Latest consensus state: current_term: 1 leader_uuid: "b21a55a004b44b57ab54ce49daadff96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21a55a004b44b57ab54ce49daadff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } } }
I20260812 07:01:37.177168 31065 sys_catalog.cc:458] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:37.183539 31073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:37.202625 31073 catalog_manager.cc:1347] Generated new cluster ID: 0ba44dfbbfc14959a514a5ed9dba8a41
I20260812 07:01:37.202766 31073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:37.217267 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:37.190638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:55247 (local address 127.0.0.1:38053)
0812 07:01:37.191635 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:37.191644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:37.199922 (+ 8278us) server_negotiation.cc:407] Connection header received
0812 07:01:37.200273 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:37.200281 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:37.200377 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:37.200453 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:37.202766 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:37.203932 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:37.206068 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:37.206734 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:37.206821 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:37.210157 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:37.210164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:37.210169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:37.210207 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:37.213809 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:37.214735 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:37.214740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:37.214744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:37.214893 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:37.215388 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:37.215393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:37.215397 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:37.215787 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:37.216026 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:37.216710 (+ 684us) server_negotiation.cc:299] Negotiation successful
0812 07:01:37.216878 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":818}
I20260812 07:01:37.222949 31073 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:37.224452 31073 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:37.231292 31073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b21a55a004b44b57ab54ce49daadff96: Generated new TSK 0
I20260812 07:01:37.232194 31073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:37.242117 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 30951
I20260812 07:01:37.243949 31045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:37.352003 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 30951
W20260812 07:01:37.380751 30930 connection.cc:535] client connection to 127.0.0.1:38053 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38053): Connection reset by peer (error 104)
2026-08-12T07:01:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:37 dist-test-slave-t3q3 krb5kdc[31103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:37 dist-test-slave-t3q3 krb5kdc[31103](info): set up 2 sockets
Aug 12 07:01:37 dist-test-slave-t3q3 krb5kdc[31103](info): commencing operation
krb5kdc: starting...
W20260812 07:01:38.764315 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.364s user 0.009s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:38 dist-test-slave-t3q3 krb5kdc[31103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:39.129587 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:39.494863 31122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:39.495271 31122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:39.495461 31122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:39.503750 31122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:39.503932 31122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:39.504073 31122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:39.504192 31122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:39.504272 31122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:39.514653 31122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39335
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:39.516237 31122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:39.517656 31122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:39.530196 31130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:39.530161 31136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:39.530269 31134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:39.536475 31122 server_base.cc:1034] running on GCE node
I20260812 07:01:39.537425 31122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:39.539024 31122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:39.540267 31122 hybrid_clock.cc:648] HybridClock initialized: now 1786518099540215 us; error 34 us; skew 500 ppm
I20260812 07:01:39.540644 31122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:39 dist-test-slave-t3q3 krb5kdc[31103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518099, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:39.544001 31122 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:39.545827 31122 webserver.cc:460] Webserver started at http://127.0.0.1:40029/ using document root <none> and password file <none>
I20260812 07:01:39.546587 31122 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:39.546751 31122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:39.547087 31122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:39.550266 31122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "597c63e8af2a4cccb69484d69a1b2e02"
format_stamp: "Formatted at 2026-08-12 07:01:39 on dist-test-slave-t3q3"
server_key: "832c9b78eec122c3da1864c255d367da"
server_key_iv: "07aeab4457cfd571631baf65556055c8"
server_key_version: "encryptionkey@0"
I20260812 07:01:39.551066 31122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "597c63e8af2a4cccb69484d69a1b2e02"
format_stamp: "Formatted at 2026-08-12 07:01:39 on dist-test-slave-t3q3"
server_key: "832c9b78eec122c3da1864c255d367da"
server_key_iv: "07aeab4457cfd571631baf65556055c8"
server_key_version: "encryptionkey@0"
I20260812 07:01:39.556488 31122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 07:01:39.559959 31149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:39.561393 31122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:01:39.561604 31122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "597c63e8af2a4cccb69484d69a1b2e02"
format_stamp: "Formatted at 2026-08-12 07:01:39 on dist-test-slave-t3q3"
server_key: "832c9b78eec122c3da1864c255d367da"
server_key_iv: "07aeab4457cfd571631baf65556055c8"
server_key_version: "encryptionkey@0"
I20260812 07:01:39.561842 31122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:39.595348 31122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:39.599818 31122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:39.600165 31122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:39.620149 31122 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39335
I20260812 07:01:39.620246 31233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39335 every 8 connection(s)
I20260812 07:01:39.621598 31122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:39.625403 31236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:39.626696 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 31122
I20260812 07:01:39.626890 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:39.627506 27528 external_mini_cluster.cc:1428] Setting key a906b152c4eb08e9f0324ee87ff94df0
I20260812 07:01:39.639395 31236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02: Bootstrap starting.
I20260812 07:01:39.643427 31236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:39.644237 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:39.629732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:33265 (local address 127.0.0.1:39335)
0812 07:01:39.630845 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:39.630866 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:39.631269 (+ 403us) server_negotiation.cc:407] Connection header received
0812 07:01:39.633785 (+ 2516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:39.633816 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:39.634322 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:39.634773 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:39.635318 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:39.637481 (+ 2163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:39.638775 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:39.639847 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:39.640042 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:39.641320 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:39.641330 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:39.641349 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:39.642072 (+ 723us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:39.642186 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:39.642224 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:39.642397 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:39.642623 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 07:01:39.642795 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":467,"threads_started":1}
I20260812 07:01:39.644820 31236 log.cc:824] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:39.648177 31236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02: No bootstrap required, opened a new log
I20260812 07:01:39.648698 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:39.658725 31236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597c63e8af2a4cccb69484d69a1b2e02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39335 } }
I20260812 07:01:39.659252 31236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:39.659389 31236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 597c63e8af2a4cccb69484d69a1b2e02, State: Initialized, Role: FOLLOWER
I20260812 07:01:39.660123 31236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597c63e8af2a4cccb69484d69a1b2e02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39335 } }
I20260812 07:01:39.660419 31236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:39.660627 31236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:39.660858 31236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:39.661461 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:39.654323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:35641 (local address 127.0.0.1:39335)
0812 07:01:39.654503 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:39.654511 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:39.655012 (+ 501us) server_negotiation.cc:407] Connection header received
0812 07:01:39.655178 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:39.655183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:39.655272 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:39.655382 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:39.655857 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:39.657069 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:39.658436 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:39.659105 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:39.659254 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:39.660658 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:39.660665 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:39.660671 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:39.660694 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:39.660744 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:39.660767 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:39.660909 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:39.661117 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 07:01:39.661173 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 07:01:39.663769 31236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597c63e8af2a4cccb69484d69a1b2e02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39335 } }
I20260812 07:01:39.664408 31236 leader_election.cc:302] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 597c63e8af2a4cccb69484d69a1b2e02; no voters:
I20260812 07:01:39.665141 31236 leader_election.cc:288] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:39.665300 31258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:39.666011 31258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [term 1 LEADER]: Becoming Leader. State: Replica: 597c63e8af2a4cccb69484d69a1b2e02, State: Running, Role: LEADER
I20260812 07:01:39.666672 31258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597c63e8af2a4cccb69484d69a1b2e02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39335 } }
I20260812 07:01:39.667562 31236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:39.669855 31260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 597c63e8af2a4cccb69484d69a1b2e02. Latest consensus state: current_term: 1 leader_uuid: "597c63e8af2a4cccb69484d69a1b2e02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597c63e8af2a4cccb69484d69a1b2e02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39335 } } }
I20260812 07:01:39.670279 31260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:39.670876 31259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "597c63e8af2a4cccb69484d69a1b2e02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597c63e8af2a4cccb69484d69a1b2e02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39335 } } }
I20260812 07:01:39.671268 31259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:39.672909 31269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:39.679312 31269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:39.689016 31269 catalog_manager.cc:1347] Generated new cluster ID: 5917136933e74905b7e3444ed741b117
I20260812 07:01:39.689163 31269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:39.697614 31269 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:39.699397 31269 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:39.710922 31269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 597c63e8af2a4cccb69484d69a1b2e02: Generated new TSK 0
I20260812 07:01:39.711906 31269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:39.721136 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31122
W20260812 07:01:39.722291 31223 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.50:33265
W20260812 07:01:39.723093 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31122: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:39.723225 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 31122
W20260812 07:01:39.748311 31100 connection.cc:535] client connection to 127.0.0.1:39335 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39335): Connection reset by peer (error 104)
2026-08-12T07:01:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2367 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:39 dist-test-slave-t3q3 krb5kdc[31294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:39 dist-test-slave-t3q3 krb5kdc[31294](info): set up 2 sockets
Aug 12 07:01:39 dist-test-slave-t3q3 krb5kdc[31294](info): commencing operation
krb5kdc: starting...
W20260812 07:01:41.106814 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:41 dist-test-slave-t3q3 krb5kdc[31294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:01:41.463358 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:41.818100 31316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:41.818439 31316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:41.818610 31316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:41.826062 31316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:41.826190 31316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:41.826289 31316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:41.826402 31316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:41.826483 31316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:41.837011 31316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35941
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:41.838419 31316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:41.839748 31316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:41.853080 31323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:41.853035 31326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:41.853132 31324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:41.854115 31316 server_base.cc:1034] running on GCE node
I20260812 07:01:41.855067 31316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:41.856766 31316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:41.858019 31316 hybrid_clock.cc:648] HybridClock initialized: now 1786518101857962 us; error 57 us; skew 500 ppm
I20260812 07:01:41.858439 31316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:41 dist-test-slave-t3q3 krb5kdc[31294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518101, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:41.861917 31316 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:01:41.863593 31316 webserver.cc:460] Webserver started at http://127.0.0.1:37873/ using document root <none> and password file <none>
I20260812 07:01:41.864439 31316 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:41.864609 31316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:41.864936 31316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:41.868170 31316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "6e08ba6af81947979cbf6d4848bdcd83"
format_stamp: "Formatted at 2026-08-12 07:01:41 on dist-test-slave-t3q3"
server_key: "de631af01e4e725f3d3013da6dca03d2"
server_key_iv: "57097d2c4a868ceb26c3bf5a782018d8"
server_key_version: "encryptionkey@0"
I20260812 07:01:41.868978 31316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "6e08ba6af81947979cbf6d4848bdcd83"
format_stamp: "Formatted at 2026-08-12 07:01:41 on dist-test-slave-t3q3"
server_key: "de631af01e4e725f3d3013da6dca03d2"
server_key_iv: "57097d2c4a868ceb26c3bf5a782018d8"
server_key_version: "encryptionkey@0"
I20260812 07:01:41.874399 31316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 07:01:41.877935 31340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:41.879230 31316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:01:41.879424 31316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "6e08ba6af81947979cbf6d4848bdcd83"
format_stamp: "Formatted at 2026-08-12 07:01:41 on dist-test-slave-t3q3"
server_key: "de631af01e4e725f3d3013da6dca03d2"
server_key_iv: "57097d2c4a868ceb26c3bf5a782018d8"
server_key_version: "encryptionkey@0"
I20260812 07:01:41.879657 31316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:41.885689 31316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:41.890293 31316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:41.890609 31316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:41.909866 31316 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35941
I20260812 07:01:41.909958 31423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35941 every 8 connection(s)
I20260812 07:01:41.911391 31316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:41.915226 31424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:41.918430 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 31316
I20260812 07:01:41.918608 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:41.919205 27528 external_mini_cluster.cc:1428] Setting key f44930da34645875171a39f047e029f8
I20260812 07:01:41.927578 31429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:41.921688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:43839 (local address 127.0.0.1:35941)
0812 07:01:41.922486 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:41.922498 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:41.922538 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:01:41.924246 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:41.924268 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:41.924612 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:41.924932 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:41.926056 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:41.926063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:41.926075 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:41.926102 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:41.926189 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:41.926209 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:41.926388 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:41.926497 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 07:01:41.926689 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":373,"threads_started":1}
I20260812 07:01:41.928341 31424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83: Bootstrap starting.
I20260812 07:01:41.931239 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:41.931555 31424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:41.932849 31424 log.cc:824] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:41.936336 31424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83: No bootstrap required, opened a new log
I20260812 07:01:41.939435 31429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:41.936652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:60129 (local address 127.0.0.1:35941)
0812 07:01:41.936863 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:41.936869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:41.937401 (+ 532us) server_negotiation.cc:407] Connection header received
0812 07:01:41.937547 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:41.937554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:41.937613 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:41.937730 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:41.938718 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:41.938724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:41.938731 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:41.938750 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:41.938798 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:41.938821 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:41.938939 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:41.939054 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 07:01:41.939125 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:01:41.944630 31424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e08ba6af81947979cbf6d4848bdcd83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } }
I20260812 07:01:41.944995 31424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:41.945072 31424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e08ba6af81947979cbf6d4848bdcd83, State: Initialized, Role: FOLLOWER
I20260812 07:01:41.945566 31424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e08ba6af81947979cbf6d4848bdcd83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } }
I20260812 07:01:41.945791 31424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:41.945889 31424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:41.946018 31424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:41.948472 31424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e08ba6af81947979cbf6d4848bdcd83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } }
I20260812 07:01:41.948927 31424 leader_election.cc:302] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e08ba6af81947979cbf6d4848bdcd83; no voters:
I20260812 07:01:41.949532 31424 leader_election.cc:288] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:41.949717 31439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:41.950448 31439 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [term 1 LEADER]: Becoming Leader. State: Replica: 6e08ba6af81947979cbf6d4848bdcd83, State: Running, Role: LEADER
I20260812 07:01:41.951050 31439 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e08ba6af81947979cbf6d4848bdcd83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } }
I20260812 07:01:41.951647 31424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:41.954190 31441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e08ba6af81947979cbf6d4848bdcd83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e08ba6af81947979cbf6d4848bdcd83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } } }
I20260812 07:01:41.954736 31441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:41.954576 31442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e08ba6af81947979cbf6d4848bdcd83. Latest consensus state: current_term: 1 leader_uuid: "6e08ba6af81947979cbf6d4848bdcd83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e08ba6af81947979cbf6d4848bdcd83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } } }
I20260812 07:01:41.954922 31442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:41.956825 31451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:41.962124 31451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:41.971652 31451 catalog_manager.cc:1347] Generated new cluster ID: 4a909d3ada914564ab4abf0b86726cea
I20260812 07:01:41.971804 31451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:41.990238 31451 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:41.990733 31451 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:41.998883 31451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e08ba6af81947979cbf6d4848bdcd83: Generated new TSK 0
I20260812 07:01:41.999847 31451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:42.013375 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31316
W20260812 07:01:42.014673 31414 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.50:43839
W20260812 07:01:42.015426 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31316: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:42.015583 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 31316
2026-08-12T07:01:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:42 dist-test-slave-t3q3 krb5kdc[31475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:42 dist-test-slave-t3q3 krb5kdc[31475](info): set up 2 sockets
Aug 12 07:01:42 dist-test-slave-t3q3 krb5kdc[31475](info): commencing operation
krb5kdc: starting...
W20260812 07:01:43.403458 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.342s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:43 dist-test-slave-t3q3 krb5kdc[31475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:01:43.760178 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:39049
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:38995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:39049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:44.125908 31496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:44.126202 31496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:44.126297 31496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:44.133431 31496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:44.133546 31496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:44.133594 31496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:44.133654 31496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:44.133700 31496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:44.143404 31496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:38995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:39049
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:39049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:44.144656 31496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:44.146016 31496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:44.157982 31509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:44.158053 31506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:44.157881 31511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:44.159276 31496 server_base.cc:1034] running on GCE node
I20260812 07:01:44.160207 31496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:44.161695 31496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:44.162945 31496 hybrid_clock.cc:648] HybridClock initialized: now 1786518104162881 us; error 49 us; skew 500 ppm
I20260812 07:01:44.163327 31496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:44 dist-test-slave-t3q3 krb5kdc[31475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518104, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:44.166638 31496 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:01:44.168218 31496 webserver.cc:460] Webserver started at http://127.26.226.62:37197/ using document root <none> and password file <none>
I20260812 07:01:44.168936 31496 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:44.169096 31496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:44.169414 31496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:44.172495 31496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "f5bac3542a904930b6d71e0638ac09ad"
format_stamp: "Formatted at 2026-08-12 07:01:44 on dist-test-slave-t3q3"
server_key: "81bd2bdb1409026ac22cf85e59f9f271"
server_key_iv: "e6058bd56afbd4be22bad0338552b6e6"
server_key_version: "encryptionkey@0"
I20260812 07:01:44.173302 31496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "f5bac3542a904930b6d71e0638ac09ad"
format_stamp: "Formatted at 2026-08-12 07:01:44 on dist-test-slave-t3q3"
server_key: "81bd2bdb1409026ac22cf85e59f9f271"
server_key_iv: "e6058bd56afbd4be22bad0338552b6e6"
server_key_version: "encryptionkey@0"
I20260812 07:01:44.178612 31496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:01:44.182229 31519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:44.183593 31496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:01:44.183791 31496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "f5bac3542a904930b6d71e0638ac09ad"
format_stamp: "Formatted at 2026-08-12 07:01:44 on dist-test-slave-t3q3"
server_key: "81bd2bdb1409026ac22cf85e59f9f271"
server_key_iv: "e6058bd56afbd4be22bad0338552b6e6"
server_key_version: "encryptionkey@0"
I20260812 07:01:44.183998 31496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:44.202541 31496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:44.207140 31496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:44.207453 31496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:44.226681 31496 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:39049
I20260812 07:01:44.226715 31600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:39049 every 8 connection(s)
I20260812 07:01:44.228052 31496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:44.231719 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:44.236599 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 31496
I20260812 07:01:44.236806 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:44.237310 27528 external_mini_cluster.cc:1428] Setting key ab9701f13e232840e806d27473d3d85b
I20260812 07:01:44.243988 31602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad: Bootstrap starting.
I20260812 07:01:44.247718 31602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:44.248903 31602 log.cc:824] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:44.252040 31602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad: No bootstrap required, opened a new log
Aug 12 07:01:44 dist-test-slave-t3q3 krb5kdc[31475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:44.260329 31602 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bac3542a904930b6d71e0638ac09ad" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39049 } }
I20260812 07:01:44.260738 31602 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:44.260869 31602 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5bac3542a904930b6d71e0638ac09ad, State: Initialized, Role: FOLLOWER
I20260812 07:01:44.261618 31602 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bac3542a904930b6d71e0638ac09ad" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39049 } }
I20260812 07:01:44.261919 31602 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:44.262100 31602 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:44.262291 31602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:44.262830 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:44.239259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49921 (local address 127.26.226.62:39049)
0812 07:01:44.240203 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:44.240232 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:44.240286 (+ 54us) server_negotiation.cc:407] Connection header received
0812 07:01:44.242739 (+ 2453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:44.242767 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:44.243260 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:44.243695 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:44.245248 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:44.247353 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:44.248810 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:44.249982 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:44.250234 (+ 252us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:44.254675 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:44.254687 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:44.254702 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:44.254740 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:44.258090 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:44.258972 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:44.258979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:44.258986 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:44.259154 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:44.259676 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:44.259680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:44.259682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:44.260317 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:44.260682 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:44.261233 (+ 551us) server_negotiation.cc:299] Negotiation successful
0812 07:01:44.261574 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":434,"threads_started":1}
I20260812 07:01:44.265354 31602 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bac3542a904930b6d71e0638ac09ad" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39049 } }
I20260812 07:01:44.265956 31602 leader_election.cc:302] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5bac3542a904930b6d71e0638ac09ad; no voters:
I20260812 07:01:44.266659 31602 leader_election.cc:288] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:44.266816 31610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:44.267359 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:44.268522 31610 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [term 1 LEADER]: Becoming Leader. State: Replica: f5bac3542a904930b6d71e0638ac09ad, State: Running, Role: LEADER
I20260812 07:01:44.269068 31602 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:44.269105 31610 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bac3542a904930b6d71e0638ac09ad" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39049 } }
I20260812 07:01:44.271739 31615 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5bac3542a904930b6d71e0638ac09ad. Latest consensus state: current_term: 1 leader_uuid: "f5bac3542a904930b6d71e0638ac09ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bac3542a904930b6d71e0638ac09ad" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39049 } } }
I20260812 07:01:44.272172 31615 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:44.272749 31611 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5bac3542a904930b6d71e0638ac09ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bac3542a904930b6d71e0638ac09ad" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39049 } } }
I20260812 07:01:44.273075 31611 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:44.278910 31619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:44.286550 31619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:44.308974 31619 catalog_manager.cc:1347] Generated new cluster ID: 35724cc089004a35b8a02dffa77bb80c
I20260812 07:01:44.309116 31619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:44.325019 31619 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:44.326854 31619 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:44.339759 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:44.310158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59411 (local address 127.26.226.62:39049)
0812 07:01:44.310348 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:44.310354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:44.320697 (+ 10343us) server_negotiation.cc:407] Connection header received
0812 07:01:44.321218 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:44.321224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:44.321302 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:44.321370 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:44.323560 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:44.324702 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:44.326515 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:44.327185 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:44.327308 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:44.330558 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:44.330566 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:44.330572 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:44.330614 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:44.334780 (+ 4166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:44.336103 (+ 1323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:44.336111 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:44.336115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:44.336297 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:44.336963 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:44.336969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:44.336974 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:44.337395 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:44.337681 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:44.339083 (+ 1402us) server_negotiation.cc:299] Negotiation successful
0812 07:01:44.339283 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":76}
I20260812 07:01:44.351086 31619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5bac3542a904930b6d71e0638ac09ad: Generated new TSK 0
I20260812 07:01:44.352432 31619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:44.359700 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31496
I20260812 07:01:44.361426 31594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:44.452389 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 31496
W20260812 07:01:44.480629 31468 connection.cc:535] client connection to 127.26.226.62:39049 recv error: Network error: failed to read from TLS socket (remote: 127.26.226.62:39049): Connection reset by peer (error 104)
2026-08-12T07:01:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:44 dist-test-slave-t3q3 krb5kdc[31652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:44 dist-test-slave-t3q3 krb5kdc[31652](info): set up 2 sockets
Aug 12 07:01:44 dist-test-slave-t3q3 krb5kdc[31652](info): commencing operation
krb5kdc: starting...
W20260812 07:01:45.839351 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:46 dist-test-slave-t3q3 krb5kdc[31652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:01:46.209507 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:42725
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:39313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:42725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:46.568862 31674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:46.569164 31674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:46.569339 31674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:46.576633 31674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:46.576757 31674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:46.576881 31674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:46.577000 31674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:46.577064 31674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:46.587011 31674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:39313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:42725
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:42725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:46.588392 31674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:46.589756 31674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:46.601892 31682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:46.601920 31683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:46.601831 31685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:46.602726 31674 server_base.cc:1034] running on GCE node
I20260812 07:01:46.603585 31674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:46.605144 31674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:46.606382 31674 hybrid_clock.cc:648] HybridClock initialized: now 1786518106606342 us; error 39 us; skew 500 ppm
I20260812 07:01:46.606751 31674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:46 dist-test-slave-t3q3 krb5kdc[31652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518106, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:46.614179 31674 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:01:46.615825 31674 webserver.cc:460] Webserver started at http://127.26.226.62:36053/ using document root <none> and password file <none>
I20260812 07:01:46.616596 31674 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:46.616758 31674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:46.617080 31674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:46.620239 31674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "22880e4db3a347b5a1a9e9e49eb111d0"
format_stamp: "Formatted at 2026-08-12 07:01:46 on dist-test-slave-t3q3"
server_key: "2b43873429d7c783753fee3041ac9908"
server_key_iv: "7d4c93e352a22e00022ddf0a4bdc0d7f"
server_key_version: "encryptionkey@0"
I20260812 07:01:46.621047 31674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "22880e4db3a347b5a1a9e9e49eb111d0"
format_stamp: "Formatted at 2026-08-12 07:01:46 on dist-test-slave-t3q3"
server_key: "2b43873429d7c783753fee3041ac9908"
server_key_iv: "7d4c93e352a22e00022ddf0a4bdc0d7f"
server_key_version: "encryptionkey@0"
I20260812 07:01:46.626616 31674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:01:46.630498 31699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:46.631860 31674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 07:01:46.632099 31674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "22880e4db3a347b5a1a9e9e49eb111d0"
format_stamp: "Formatted at 2026-08-12 07:01:46 on dist-test-slave-t3q3"
server_key: "2b43873429d7c783753fee3041ac9908"
server_key_iv: "7d4c93e352a22e00022ddf0a4bdc0d7f"
server_key_version: "encryptionkey@0"
I20260812 07:01:46.632331 31674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:46.651435 31674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:46.656512 31674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:46.656844 31674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:46.678068 31674 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:42725
I20260812 07:01:46.678128 31780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:42725 every 8 connection(s)
I20260812 07:01:46.679563 31674 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:46.683414 31781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:46.685717 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 31674
I20260812 07:01:46.685904 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:46.686460 27528 external_mini_cluster.cc:1428] Setting key 0169ad1e03fdeda95f15c41a6b86b322
I20260812 07:01:46.695595 31781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0: Bootstrap starting.
I20260812 07:01:46.698832 31781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:46.700264 31781 log.cc:824] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:46.700328 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:46.688558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50313 (local address 127.26.226.62:42725)
0812 07:01:46.689390 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:46.689402 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:46.689444 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:01:46.691168 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:46.691189 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:46.691582 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:46.691973 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:46.692588 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:46.693986 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:46.695367 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:46.696289 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:46.696487 (+ 198us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:46.697818 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:46.697826 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:46.697841 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:46.698467 (+ 626us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:46.698546 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:46.698576 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:46.698758 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:46.699060 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 07:01:46.699269 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":348,"threads_started":1}
I20260812 07:01:46.704404 31781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0: No bootstrap required, opened a new log
I20260812 07:01:46.705339 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:46.715612 31781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22880e4db3a347b5a1a9e9e49eb111d0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42725 } }
I20260812 07:01:46.716224 31781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:46.716351 31781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22880e4db3a347b5a1a9e9e49eb111d0, State: Initialized, Role: FOLLOWER
I20260812 07:01:46.717123 31781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22880e4db3a347b5a1a9e9e49eb111d0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42725 } }
I20260812 07:01:46.717449 31781 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:46.717626 31781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:46.717868 31781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:46.717945 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:46.710790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58765 (local address 127.26.226.62:42725)
0812 07:01:46.711065 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:46.711071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:46.711461 (+ 390us) server_negotiation.cc:407] Connection header received
0812 07:01:46.711646 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:46.711650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:46.711713 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:46.711802 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:46.712447 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:46.713261 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:46.715209 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:46.715664 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:46.715762 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:46.717038 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:46.717043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:46.717046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:46.717063 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:46.717126 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:46.717143 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:46.717231 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:46.717557 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 07:01:46.717621 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 07:01:46.720678 31781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22880e4db3a347b5a1a9e9e49eb111d0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42725 } }
I20260812 07:01:46.721232 31781 leader_election.cc:302] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22880e4db3a347b5a1a9e9e49eb111d0; no voters:
I20260812 07:01:46.721848 31781 leader_election.cc:288] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:46.721974 31799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:46.722708 31799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [term 1 LEADER]: Becoming Leader. State: Replica: 22880e4db3a347b5a1a9e9e49eb111d0, State: Running, Role: LEADER
I20260812 07:01:46.723313 31799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22880e4db3a347b5a1a9e9e49eb111d0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42725 } }
I20260812 07:01:46.724146 31781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:46.726089 31802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22880e4db3a347b5a1a9e9e49eb111d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22880e4db3a347b5a1a9e9e49eb111d0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42725 } } }
I20260812 07:01:46.726496 31802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:46.726758 31803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22880e4db3a347b5a1a9e9e49eb111d0. Latest consensus state: current_term: 1 leader_uuid: "22880e4db3a347b5a1a9e9e49eb111d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22880e4db3a347b5a1a9e9e49eb111d0" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42725 } } }
I20260812 07:01:46.727180 31803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:46.736886 31809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:46.744129 31809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:46.754112 31809 catalog_manager.cc:1347] Generated new cluster ID: 9b6612bb1af640b1bd81ce56283745b2
I20260812 07:01:46.754256 31809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:46.788772 31809 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:46.790737 31809 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:46.799209 31809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22880e4db3a347b5a1a9e9e49eb111d0: Generated new TSK 0
I20260812 07:01:46.800175 31809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:46.807386 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31674
W20260812 07:01:46.808719 31773 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50313
W20260812 07:01:46.809573 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31674: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:46.809731 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 31674
W20260812 07:01:46.839059 31650 connection.cc:535] client connection to 127.26.226.62:42725 recv error: Network error: failed to read from TLS socket (remote: 127.26.226.62:42725): Connection reset by peer (error 104)
2026-08-12T07:01:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2358 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:46 dist-test-slave-t3q3 krb5kdc[31839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:46 dist-test-slave-t3q3 krb5kdc[31839](info): set up 2 sockets
Aug 12 07:01:46 dist-test-slave-t3q3 krb5kdc[31839](info): commencing operation
krb5kdc: starting...
W20260812 07:01:48.215760 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.358s user 0.004s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:48 dist-test-slave-t3q3 krb5kdc[31839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:01:48.574915 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:42489
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:42489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:48.944253 31857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:48.944813 31857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:48.945010 31857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:48.953315 31857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:48.953426 31857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:48.953547 31857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:48.953655 31857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:48.953718 31857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:48.963766 31857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:42489
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:42489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:48.965138 31857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:48.966517 31857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:48.979017 31862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:48.978969 31865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:48.979055 31867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:48.980149 31857 server_base.cc:1034] running on GCE node
I20260812 07:01:48.980986 31857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:48.982424 31857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:48.983642 31857 hybrid_clock.cc:648] HybridClock initialized: now 1786518108983601 us; error 32 us; skew 500 ppm
I20260812 07:01:48.983976 31857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:48 dist-test-slave-t3q3 krb5kdc[31839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518108, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:48.987315 31857 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:01:48.988979 31857 webserver.cc:460] Webserver started at http://127.26.226.62:33697/ using document root <none> and password file <none>
I20260812 07:01:48.989698 31857 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:48.989846 31857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:48.990188 31857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:48.993439 31857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "7fad13ae12e1463c8bb96b911b79f2cb"
format_stamp: "Formatted at 2026-08-12 07:01:48 on dist-test-slave-t3q3"
server_key: "eb0fa7e5337b1587deb899178580d45f"
server_key_iv: "2a5159032573ed5940b0a97775490e05"
server_key_version: "encryptionkey@0"
I20260812 07:01:48.994194 31857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "7fad13ae12e1463c8bb96b911b79f2cb"
format_stamp: "Formatted at 2026-08-12 07:01:48 on dist-test-slave-t3q3"
server_key: "eb0fa7e5337b1587deb899178580d45f"
server_key_iv: "2a5159032573ed5940b0a97775490e05"
server_key_version: "encryptionkey@0"
I20260812 07:01:48.999454 31857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 07:01:49.003156 31875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:49.004544 31857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:49.004727 31857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "7fad13ae12e1463c8bb96b911b79f2cb"
format_stamp: "Formatted at 2026-08-12 07:01:48 on dist-test-slave-t3q3"
server_key: "eb0fa7e5337b1587deb899178580d45f"
server_key_iv: "2a5159032573ed5940b0a97775490e05"
server_key_version: "encryptionkey@0"
I20260812 07:01:49.004942 31857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:49.011032 31857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:49.015134 31857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:49.015439 31857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:49.035184 31857 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:42489
I20260812 07:01:49.035216 31957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:42489 every 8 connection(s)
I20260812 07:01:49.036777 31857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:49.040719 31959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:49.041652 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 31857
I20260812 07:01:49.041859 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:49.042341 27528 external_mini_cluster.cc:1428] Setting key c1258dcf19513fadf492b33dafaafe75
I20260812 07:01:49.051752 31963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:49.044405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58345 (local address 127.26.226.62:42489)
0812 07:01:49.045303 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:49.045316 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:49.045364 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:01:49.047590 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:49.047623 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:49.048157 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:49.048670 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:49.049856 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:49.049864 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:49.049878 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:49.049894 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:49.049960 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:49.049982 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:49.050273 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:49.050457 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 07:01:49.050663 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":416,"threads_started":1}
I20260812 07:01:49.054903 31959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb: Bootstrap starting.
I20260812 07:01:49.056109 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:49.058264 31959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:49.059450 31959 log.cc:824] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:49.062749 31959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb: No bootstrap required, opened a new log
I20260812 07:01:49.064028 31963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:49.061133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57395 (local address 127.26.226.62:42489)
0812 07:01:49.061326 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:49.061332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:49.061969 (+ 637us) server_negotiation.cc:407] Connection header received
0812 07:01:49.062131 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:49.062149 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:49.062214 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:49.062334 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:49.063349 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:49.063355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:49.063359 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:49.063380 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:49.063435 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:49.063456 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:49.063581 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:49.063687 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 07:01:49.063756 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:01:49.070705 31959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fad13ae12e1463c8bb96b911b79f2cb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42489 } }
I20260812 07:01:49.071173 31959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:49.071292 31959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fad13ae12e1463c8bb96b911b79f2cb, State: Initialized, Role: FOLLOWER
I20260812 07:01:49.071961 31959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fad13ae12e1463c8bb96b911b79f2cb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42489 } }
I20260812 07:01:49.072249 31959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:49.072362 31959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:49.072484 31959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:49.074853 31959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fad13ae12e1463c8bb96b911b79f2cb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42489 } }
I20260812 07:01:49.075429 31959 leader_election.cc:302] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fad13ae12e1463c8bb96b911b79f2cb; no voters:
I20260812 07:01:49.076035 31959 leader_election.cc:288] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:49.076260 31974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:49.076995 31974 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [term 1 LEADER]: Becoming Leader. State: Replica: 7fad13ae12e1463c8bb96b911b79f2cb, State: Running, Role: LEADER
I20260812 07:01:49.077615 31974 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fad13ae12e1463c8bb96b911b79f2cb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42489 } }
I20260812 07:01:49.078408 31959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:49.080893 31977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fad13ae12e1463c8bb96b911b79f2cb. Latest consensus state: current_term: 1 leader_uuid: "7fad13ae12e1463c8bb96b911b79f2cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fad13ae12e1463c8bb96b911b79f2cb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42489 } } }
I20260812 07:01:49.081411 31977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:49.080941 31975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fad13ae12e1463c8bb96b911b79f2cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fad13ae12e1463c8bb96b911b79f2cb" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42489 } } }
I20260812 07:01:49.082468 31975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:49.085136 31985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:49.090207 31985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:49.099753 31985 catalog_manager.cc:1347] Generated new cluster ID: 827df85b4eb54872a7de30b5767f6e0a
I20260812 07:01:49.099903 31985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:49.122023 31985 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:49.122543 31985 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:49.129761 31985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fad13ae12e1463c8bb96b911b79f2cb: Generated new TSK 0
I20260812 07:01:49.130808 31985 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:49.141804 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31857
W20260812 07:01:49.143527 31951 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58345
W20260812 07:01:49.144336 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 31857: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:49.144511 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 31857
2026-08-12T07:01:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:49 dist-test-slave-t3q3 krb5kdc[32014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:49 dist-test-slave-t3q3 krb5kdc[32014](info): set up 2 sockets
Aug 12 07:01:49 dist-test-slave-t3q3 krb5kdc[32014](info): commencing operation
krb5kdc: starting...
W20260812 07:01:50.534090 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.341s user 0.001s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:50 dist-test-slave-t3q3 krb5kdc[32014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:01:50.903458 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:39881
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:39881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:51.262859 32034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:51.263170 32034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:51.263296 32034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:51.270516 32034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:51.270629 32034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:51.270677 32034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:51.270741 32034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:51.270787 32034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:51.280519 32034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:39881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:39881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:51.281831 32034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:51.283097 32034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:51.295459 32044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:51.295461 32045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:51.295710 32047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:51.296478 32034 server_base.cc:1034] running on GCE node
I20260812 07:01:51.297353 32034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:51.299033 32034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:51.300288 32034 hybrid_clock.cc:648] HybridClock initialized: now 1786518111300238 us; error 65 us; skew 500 ppm
I20260812 07:01:51.300694 32034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:51 dist-test-slave-t3q3 krb5kdc[32014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518111, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:51.304265 32034 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:01:51.305964 32034 webserver.cc:460] Webserver started at http://127.26.226.62:40645/ using document root <none> and password file <none>
I20260812 07:01:51.306717 32034 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:51.306880 32034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:51.307214 32034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:51.310552 32034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "f8e7fcf887f04facb2d9681b18b2ddba"
format_stamp: "Formatted at 2026-08-12 07:01:51 on dist-test-slave-t3q3"
server_key: "55e43ef2bfe5dd33172c36ea8a5e515f"
server_key_iv: "5aadf9c5b3fa086b6661373910546740"
server_key_version: "encryptionkey@0"
I20260812 07:01:51.311378 32034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "f8e7fcf887f04facb2d9681b18b2ddba"
format_stamp: "Formatted at 2026-08-12 07:01:51 on dist-test-slave-t3q3"
server_key: "55e43ef2bfe5dd33172c36ea8a5e515f"
server_key_iv: "5aadf9c5b3fa086b6661373910546740"
server_key_version: "encryptionkey@0"
I20260812 07:01:51.317026 32034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 07:01:51.321096 32058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:51.322594 32034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260812 07:01:51.322810 32034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "f8e7fcf887f04facb2d9681b18b2ddba"
format_stamp: "Formatted at 2026-08-12 07:01:51 on dist-test-slave-t3q3"
server_key: "55e43ef2bfe5dd33172c36ea8a5e515f"
server_key_iv: "5aadf9c5b3fa086b6661373910546740"
server_key_version: "encryptionkey@0"
I20260812 07:01:51.323046 32034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:51.340130 32034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:51.344444 32034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:51.344761 32034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:51.364866 32034 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:39881
I20260812 07:01:51.364895 32134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:39881 every 8 connection(s)
I20260812 07:01:51.366330 32034 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:51.369935 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 32034
I20260812 07:01:51.370173 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:51.370513 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:51.370731 27528 external_mini_cluster.cc:1428] Setting key 7fce14d895cff7193d061cc0a0747b75
I20260812 07:01:51.386731 32135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba: Bootstrap starting.
Aug 12 07:01:51 dist-test-slave-t3q3 krb5kdc[32014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:51.390741 32135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:51.392195 32135 log.cc:824] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:51.395954 32135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba: No bootstrap required, opened a new log
I20260812 07:01:51.397742 32140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:51.372614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49009 (local address 127.26.226.62:39881)
0812 07:01:51.373541 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:51.373558 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:51.373613 (+ 55us) server_negotiation.cc:407] Connection header received
0812 07:01:51.376051 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:51.376104 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:51.376665 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:51.377204 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:51.378427 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:51.380702 (+ 2275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:51.382047 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:51.383239 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:51.383458 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:51.388233 (+ 4775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:51.388247 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:51.388269 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:51.388343 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:51.392590 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:51.393566 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:51.393573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:51.393580 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:51.393785 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:51.394261 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:51.394266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:51.394269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:51.395013 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:51.395322 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:51.395800 (+ 478us) server_negotiation.cc:299] Negotiation successful
0812 07:01:51.396306 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":375,"threads_started":1}
I20260812 07:01:51.402078 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:51.405042 32135 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7fcf887f04facb2d9681b18b2ddba" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39881 } }
I20260812 07:01:51.405431 32135 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:51.405562 32135 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e7fcf887f04facb2d9681b18b2ddba, State: Initialized, Role: FOLLOWER
I20260812 07:01:51.406141 32135 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7fcf887f04facb2d9681b18b2ddba" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39881 } }
I20260812 07:01:51.406400 32135 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:51.406587 32135 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:51.406812 32135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:51.410336 32135 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7fcf887f04facb2d9681b18b2ddba" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39881 } }
I20260812 07:01:51.410933 32135 leader_election.cc:302] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8e7fcf887f04facb2d9681b18b2ddba; no voters:
I20260812 07:01:51.411612 32135 leader_election.cc:288] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:51.411782 32150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:51.413550 32150 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [term 1 LEADER]: Becoming Leader. State: Replica: f8e7fcf887f04facb2d9681b18b2ddba, State: Running, Role: LEADER
I20260812 07:01:51.414162 32135 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:51.414266 32150 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7fcf887f04facb2d9681b18b2ddba" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39881 } }
I20260812 07:01:51.417482 32151 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8e7fcf887f04facb2d9681b18b2ddba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7fcf887f04facb2d9681b18b2ddba" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39881 } } }
I20260812 07:01:51.417899 32151 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:51.418820 32161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:51.419198 32152 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8e7fcf887f04facb2d9681b18b2ddba. Latest consensus state: current_term: 1 leader_uuid: "f8e7fcf887f04facb2d9681b18b2ddba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7fcf887f04facb2d9681b18b2ddba" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 39881 } } }
I20260812 07:01:51.427929 32152 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:51.427822 32140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:51.406807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60553 (local address 127.26.226.62:39881)
0812 07:01:51.407027 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:51.407036 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:51.407603 (+ 567us) server_negotiation.cc:407] Connection header received
0812 07:01:51.407797 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:51.407801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:51.407870 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:51.407956 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:51.409912 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:51.410727 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:51.412717 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:51.413474 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:51.413638 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:51.417355 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:51.417365 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:51.417372 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:51.417416 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:51.422697 (+ 5281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:51.423935 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:51.423943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:51.423950 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:51.424153 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:51.424731 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:51.424738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:51.424744 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:51.425271 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:51.426892 (+ 1621us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:51.427075 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 07:01:51.427385 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:01:51.431540 32161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:51.441524 32161 catalog_manager.cc:1347] Generated new cluster ID: 14926afea82c485ca515ac055e72b55d
I20260812 07:01:51.441651 32161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:51.451828 32161 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:51.453483 32161 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:51.464744 32161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8e7fcf887f04facb2d9681b18b2ddba: Generated new TSK 0
I20260812 07:01:51.465867 32161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:51.471899 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32034
I20260812 07:01:51.473202 32130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:01:51.564553 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 32034
2026-08-12T07:01:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2413 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:51 dist-test-slave-t3q3 krb5kdc[32187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:51 dist-test-slave-t3q3 krb5kdc[32187](info): set up 2 sockets
Aug 12 07:01:51 dist-test-slave-t3q3 krb5kdc[32187](info): commencing operation
krb5kdc: starting...
W20260812 07:01:52.954421 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:53 dist-test-slave-t3q3 krb5kdc[32187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:01:53.328462 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:41811
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:44955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:41811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:53.693578 32206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:53.693975 32206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:53.694180 32206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:53.701555 32206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:53.701753 32206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:53.701853 32206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:53.701948 32206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:53.702018 32206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:53.712298 32206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:44955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:41811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:53.713840 32206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:53.715265 32206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:53.727882 32225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:53.727774 32222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:53.727793 32220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:53.728574 32206 server_base.cc:1034] running on GCE node
I20260812 07:01:53.729450 32206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:53.730898 32206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:53.732170 32206 hybrid_clock.cc:648] HybridClock initialized: now 1786518113732086 us; error 53 us; skew 500 ppm
I20260812 07:01:53.732493 32206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:53 dist-test-slave-t3q3 krb5kdc[32187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518113, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:53.735844 32206 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:01:53.737524 32206 webserver.cc:460] Webserver started at http://127.26.226.62:45829/ using document root <none> and password file <none>
I20260812 07:01:53.738277 32206 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:53.738421 32206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:53.738765 32206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:53.742220 32206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "8f3420666b5c4f8c928459f1b9f4fe66"
format_stamp: "Formatted at 2026-08-12 07:01:53 on dist-test-slave-t3q3"
server_key: "6aa931f9b54f0e1f566edf71013d5887"
server_key_iv: "2e3ab31fca71f5b1080a8ab83d4377ba"
server_key_version: "encryptionkey@0"
I20260812 07:01:53.743049 32206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "8f3420666b5c4f8c928459f1b9f4fe66"
format_stamp: "Formatted at 2026-08-12 07:01:53 on dist-test-slave-t3q3"
server_key: "6aa931f9b54f0e1f566edf71013d5887"
server_key_iv: "2e3ab31fca71f5b1080a8ab83d4377ba"
server_key_version: "encryptionkey@0"
I20260812 07:01:53.748821 32206 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 07:01:53.752679 32236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:53.754158 32206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:01:53.754369 32206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "8f3420666b5c4f8c928459f1b9f4fe66"
format_stamp: "Formatted at 2026-08-12 07:01:53 on dist-test-slave-t3q3"
server_key: "6aa931f9b54f0e1f566edf71013d5887"
server_key_iv: "2e3ab31fca71f5b1080a8ab83d4377ba"
server_key_version: "encryptionkey@0"
I20260812 07:01:53.754597 32206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:53.777007 32206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:53.781693 32206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:53.782033 32206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:53.802875 32206 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:41811
I20260812 07:01:53.802896 32323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:41811 every 8 connection(s)
I20260812 07:01:53.804406 32206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:53.806607 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 32206
I20260812 07:01:53.806803 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:53.807279 27528 external_mini_cluster.cc:1428] Setting key 40831bd39f6524357c44f55b2b1772ad
I20260812 07:01:53.808630 32325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:53.822866 32329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:53.809143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47475 (local address 127.26.226.62:41811)
0812 07:01:53.809976 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:53.809995 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:53.810053 (+ 58us) server_negotiation.cc:407] Connection header received
0812 07:01:53.812206 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:53.812234 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:53.812767 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:53.813257 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:53.813765 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:53.815784 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:53.817234 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:53.818283 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:53.818490 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:53.819829 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:53.819845 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:53.819864 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:53.820681 (+ 817us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:53.820778 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:53.820815 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:53.820945 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:53.821199 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 07:01:53.821495 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":349,"threads_started":1}
I20260812 07:01:53.824275 32325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66: Bootstrap starting.
I20260812 07:01:53.826767 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:53.827751 32325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:53.829087 32325 log.cc:824] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:53.832871 32325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66: No bootstrap required, opened a new log
I20260812 07:01:53.839625 32329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:53.832534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44009 (local address 127.26.226.62:41811)
0812 07:01:53.832755 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:53.832761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:53.833261 (+ 500us) server_negotiation.cc:407] Connection header received
0812 07:01:53.833424 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:53.833436 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:53.833531 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:53.833663 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:53.834169 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:53.835327 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:53.836821 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:53.837550 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:53.837686 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:53.838803 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:53.838810 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:53.838815 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:53.838838 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:53.838889 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:53.838910 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:53.839018 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:53.839146 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 07:01:53.839252 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:01:53.843319 32325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3420666b5c4f8c928459f1b9f4fe66" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41811 } }
I20260812 07:01:53.843763 32325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:53.843844 32325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3420666b5c4f8c928459f1b9f4fe66, State: Initialized, Role: FOLLOWER
I20260812 07:01:53.844475 32325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3420666b5c4f8c928459f1b9f4fe66" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41811 } }
I20260812 07:01:53.844705 32325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:53.844805 32325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:53.844949 32325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:53.847522 32325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3420666b5c4f8c928459f1b9f4fe66" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41811 } }
I20260812 07:01:53.848227 32325 leader_election.cc:302] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f3420666b5c4f8c928459f1b9f4fe66; no voters:
I20260812 07:01:53.848853 32325 leader_election.cc:288] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:53.849036 32345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:53.849802 32345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [term 1 LEADER]: Becoming Leader. State: Replica: 8f3420666b5c4f8c928459f1b9f4fe66, State: Running, Role: LEADER
I20260812 07:01:53.850378 32345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3420666b5c4f8c928459f1b9f4fe66" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41811 } }
I20260812 07:01:53.851240 32325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:53.853906 32348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f3420666b5c4f8c928459f1b9f4fe66. Latest consensus state: current_term: 1 leader_uuid: "8f3420666b5c4f8c928459f1b9f4fe66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3420666b5c4f8c928459f1b9f4fe66" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41811 } } }
I20260812 07:01:53.853883 32346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f3420666b5c4f8c928459f1b9f4fe66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3420666b5c4f8c928459f1b9f4fe66" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 41811 } } }
I20260812 07:01:53.854440 32348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:53.854493 32346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:53.861121 32353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:53.866554 32353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:53.876318 32353 catalog_manager.cc:1347] Generated new cluster ID: 997a7b98f1a141fa90b184c3845a48ce
I20260812 07:01:53.876494 32353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:53.892143 32353 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:53.894209 32353 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:53.906232 32353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f3420666b5c4f8c928459f1b9f4fe66: Generated new TSK 0
I20260812 07:01:53.907156 32353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:53.918890 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32206
W20260812 07:01:53.920241 32316 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47475
W20260812 07:01:53.921113 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32206: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:53.921281 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 32206
2026-08-12T07:01:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2358 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:54 dist-test-slave-t3q3 krb5kdc[32377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:54 dist-test-slave-t3q3 krb5kdc[32377](info): set up 2 sockets
Aug 12 07:01:54 dist-test-slave-t3q3 krb5kdc[32377](info): commencing operation
krb5kdc: starting...
W20260812 07:01:55.359788 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.394s user 0.005s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:55 dist-test-slave-t3q3 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:01:55.723939 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:35415
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:37841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:35415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:56.084750 32393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:56.085067 32393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:56.085253 32393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:56.092621 32393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:56.092756 32393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:56.092888 32393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:56.093026 32393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:56.093106 32393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:56.103449 32393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:37841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:35415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:35415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:56.104985 32393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:56.106431 32393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:56.119196 32403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:56.119239 32405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:56.119122 32399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:56.120208 32393 server_base.cc:1034] running on GCE node
I20260812 07:01:56.120991 32393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:56.122414 32393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:56.123628 32393 hybrid_clock.cc:648] HybridClock initialized: now 1786518116123591 us; error 32 us; skew 500 ppm
I20260812 07:01:56.124017 32393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:56 dist-test-slave-t3q3 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518116, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:56.127378 32393 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:01:56.129061 32393 webserver.cc:460] Webserver started at http://127.26.226.62:33397/ using document root <none> and password file <none>
I20260812 07:01:56.129832 32393 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:56.129993 32393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:56.130323 32393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:56.133651 32393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "6164fd0668784850a83eb6460d343575"
format_stamp: "Formatted at 2026-08-12 07:01:56 on dist-test-slave-t3q3"
server_key: "45c331adb641a6e456262b76f8f1fab4"
server_key_iv: "d375526f476975dc23f9ebace1c75e57"
server_key_version: "encryptionkey@0"
I20260812 07:01:56.134514 32393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "6164fd0668784850a83eb6460d343575"
format_stamp: "Formatted at 2026-08-12 07:01:56 on dist-test-slave-t3q3"
server_key: "45c331adb641a6e456262b76f8f1fab4"
server_key_iv: "d375526f476975dc23f9ebace1c75e57"
server_key_version: "encryptionkey@0"
I20260812 07:01:56.140228 32393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 07:01:56.143934 32420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:56.145447 32393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:56.145677 32393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "6164fd0668784850a83eb6460d343575"
format_stamp: "Formatted at 2026-08-12 07:01:56 on dist-test-slave-t3q3"
server_key: "45c331adb641a6e456262b76f8f1fab4"
server_key_iv: "d375526f476975dc23f9ebace1c75e57"
server_key_version: "encryptionkey@0"
I20260812 07:01:56.145910 32393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:56.152300 32393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:56.156893 32393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:56.157208 32393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:56.177417 32393 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:35415
I20260812 07:01:56.177461 32502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:35415 every 8 connection(s)
I20260812 07:01:56.178951 32393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:56.181039 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 32393
I20260812 07:01:56.181248 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:56.181797 27528 external_mini_cluster.cc:1428] Setting key 6fe91b879c6b8cce7c0c015cd2dbd09e
I20260812 07:01:56.183876 32505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:56.190153 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:56.183659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47989 (local address 127.26.226.62:35415)
0812 07:01:56.184692 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:56.184704 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:56.184773 (+ 69us) server_negotiation.cc:407] Connection header received
0812 07:01:56.186394 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:56.186417 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:56.186798 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:56.187154 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:56.188441 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:56.188447 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:56.188460 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:56.188483 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:56.188566 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:56.188587 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:56.188800 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:56.188907 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 07:01:56.189125 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":536,"threads_started":1}
I20260812 07:01:56.194669 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:56.197886 32505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575: Bootstrap starting.
I20260812 07:01:56.202306 32505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:56.202306 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:56.199486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58657 (local address 127.26.226.62:35415)
0812 07:01:56.199751 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:56.199758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:56.200167 (+ 409us) server_negotiation.cc:407] Connection header received
0812 07:01:56.200364 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:56.200368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:56.200420 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:56.200507 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:56.201576 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:56.201581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:56.201584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:01:56.201599 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:56.201642 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:01:56.201657 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:56.201756 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:56.201927 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 07:01:56.201978 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 07:01:56.203899 32505 log.cc:824] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:56.207000 32505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575: No bootstrap required, opened a new log
I20260812 07:01:56.215516 32505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6164fd0668784850a83eb6460d343575" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35415 } }
I20260812 07:01:56.216120 32505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:01:56.216259 32505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6164fd0668784850a83eb6460d343575, State: Initialized, Role: FOLLOWER
I20260812 07:01:56.216943 32505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6164fd0668784850a83eb6460d343575" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35415 } }
I20260812 07:01:56.217226 32505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:01:56.217401 32505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:01:56.217620 32505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:01:56.220181 32505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6164fd0668784850a83eb6460d343575" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35415 } }
I20260812 07:01:56.220815 32505 leader_election.cc:302] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6164fd0668784850a83eb6460d343575; no voters:
I20260812 07:01:56.221489 32505 leader_election.cc:288] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:01:56.221668 32521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:01:56.222465 32521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [term 1 LEADER]: Becoming Leader. State: Replica: 6164fd0668784850a83eb6460d343575, State: Running, Role: LEADER
I20260812 07:01:56.223078 32521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6164fd0668784850a83eb6460d343575" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35415 } }
I20260812 07:01:56.223960 32505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:01:56.230388 32523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6164fd0668784850a83eb6460d343575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6164fd0668784850a83eb6460d343575" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35415 } } }
I20260812 07:01:56.231163 32523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [sys.catalog]: This master's current role is: LEADER
I20260812 07:01:56.230412 32524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6164fd0668784850a83eb6460d343575. Latest consensus state: current_term: 1 leader_uuid: "6164fd0668784850a83eb6460d343575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6164fd0668784850a83eb6460d343575" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 35415 } } }
I20260812 07:01:56.231586 32524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575 [sys.catalog]: This master's current role is: LEADER
W20260812 07:01:56.236003 32540 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:01:56.236195 32540 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:01:56.236353 32545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:01:56.238358 32545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:01:56.247120 32545 catalog_manager.cc:1347] Generated new cluster ID: 2307142325a74f6b8e3a547f513b18ae
I20260812 07:01:56.247262 32545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:01:56.266113 32545 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:01:56.266606 32545 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:01:56.275846 32545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6164fd0668784850a83eb6460d343575: Generated new TSK 0
I20260812 07:01:56.276775 32545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:01:56.287487 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32393
W20260812 07:01:56.288681 32495 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47989
W20260812 07:01:56.289424 27528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32393: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:01:56.289589 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 32393
2026-08-12T07:01:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2372 ms)
[----------] 18 tests from AuthTokenIssuingTest (42698 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:01:56 dist-test-slave-t3q3 krb5kdc[32554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:01:56 dist-test-slave-t3q3 krb5kdc[32554](info): set up 2 sockets
Aug 12 07:01:56 dist-test-slave-t3q3 krb5kdc[32554](info): commencing operation
krb5kdc: starting...
W20260812 07:01:57.718690 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.380s user 0.000s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:01:57 dist-test-slave-t3q3 krb5kdc[32554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:01:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:01:58.091749 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:43265
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:43265,127.26.226.61:33271,127.26.226.60:32847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:01:58.455967 32575 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:01:58.456575 32575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:58.456699 32575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:58.456843 32575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:58.464759 32575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:58.464876 32575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:58.464939 32575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:58.464994 32575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:58.465075 32575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:58.475479 32575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:43265,127.26.226.61:33271,127.26.226.60:32847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:43265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:58.476936 32575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:58.478399 32575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:58.490978 32582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:58.490979 32584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:58.491163 32586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:58.491853 32575 server_base.cc:1034] running on GCE node
I20260812 07:01:58.492746 32575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:58.494243 32575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:58.495496 32575 hybrid_clock.cc:648] HybridClock initialized: now 1786518118495427 us; error 53 us; skew 500 ppm
I20260812 07:01:58.495852 32575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:58 dist-test-slave-t3q3 krb5kdc[32554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518118, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:58.499208 32575 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:01:58.500777 32575 webserver.cc:460] Webserver started at http://127.26.226.62:43791/ using document root <none> and password file <none>
I20260812 07:01:58.501485 32575 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:58.501621 32575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:58.501969 32575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:58.505095 32575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "7f2da853df8e43c685765a00db43339b"
format_stamp: "Formatted at 2026-08-12 07:01:58 on dist-test-slave-t3q3"
server_key: "e23c1a2700036f30b1cf7797b845990f"
server_key_iv: "9e9c49026ed1026d602f096d2a3cef90"
server_key_version: "encryptionkey@0"
I20260812 07:01:58.505935 32575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "7f2da853df8e43c685765a00db43339b"
format_stamp: "Formatted at 2026-08-12 07:01:58 on dist-test-slave-t3q3"
server_key: "e23c1a2700036f30b1cf7797b845990f"
server_key_iv: "9e9c49026ed1026d602f096d2a3cef90"
server_key_version: "encryptionkey@0"
I20260812 07:01:58.511370 32575 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260812 07:01:58.515407 32598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:58.517020 32575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:58.517277 32575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "7f2da853df8e43c685765a00db43339b"
format_stamp: "Formatted at 2026-08-12 07:01:58 on dist-test-slave-t3q3"
server_key: "e23c1a2700036f30b1cf7797b845990f"
server_key_iv: "9e9c49026ed1026d602f096d2a3cef90"
server_key_version: "encryptionkey@0"
I20260812 07:01:58.517537 32575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:58.534333 32575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:58.539140 32575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:58.539436 32575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:58.558974 32575 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:43265
I20260812 07:01:58.558995 32678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:43265 every 8 connection(s)
I20260812 07:01:58.560563 32575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:01:58.564450 32680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:58.569628 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 32575
I20260812 07:01:58.569871 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:01:58.570523 27528 external_mini_cluster.cc:1428] Setting key c816300d2a29451a9be55dbd926fb325
I20260812 07:01:58.571082 32680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:01:58 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518118, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:58.594938 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.574390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43265 (local address 127.0.0.1:37358)
0812 07:01:58.575571 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.575611 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:58.577405 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:58.578219 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:58.578226 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:58.578949 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:58.579294 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:58.579312 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:58.580973 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:58.580979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:58.582048 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:58.582060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:58.582997 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:58.583003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:58.583200 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:58.583896 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:58.583918 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:58.586252 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:58.590893 (+ 4641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:58.590904 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:58.590914 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:58.591334 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:58.591834 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:58.591837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:58.591839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:58.591984 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:58.593022 (+ 1038us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:58.593029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:58.593252 (+ 223us) client_negotiation.cc:769] Sending connection context
0812 07:01:58.593478 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 07:01:58.593877 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":842,"threads_started":1}
I20260812 07:01:58.594992 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.574382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37358 (local address 127.26.226.62:43265)
0812 07:01:58.575162 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:58.575172 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:58.575734 (+ 562us) server_negotiation.cc:407] Connection header received
0812 07:01:58.577769 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:58.577775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:58.577987 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:58.578185 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:58.579514 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:58.580814 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:58.582201 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:58.582758 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:58.582967 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:58.586515 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:58.586527 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:58.586545 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:58.586586 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:58.590732 (+ 4146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:58.591486 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:58.591497 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:58.591503 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:58.591716 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:58.592127 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:58.592132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:58.592138 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:58.592871 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:58.593120 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:58.593674 (+ 554us) server_negotiation.cc:299] Negotiation successful
0812 07:01:58.594079 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":581,"thread_start_us":361,"threads_started":1}
I20260812 07:01:58.598278 32680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:01:58.599367 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.598949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:35942)
0812 07:01:58.599116 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.599246 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 07:01:58.599681 32600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
I20260812 07:01:58.601372 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.600919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:35954)
0812 07:01:58.601156 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.601236 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260812 07:01:58.601660 32680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:33271: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.226.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab.
Entry for principal kudu/127.26.226.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab.
I20260812 07:01:58.649451 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.61:33271
--webserver_interface=127.26.226.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab
--principal=kudu/127.26.226.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:43265,127.26.226.61:33271,127.26.226.60:32847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:01:58.651062 32680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } attempt: 1
I20260812 07:01:58.652443 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.651806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:35970)
0812 07:01:58.652090 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.652232 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 07:01:58.654261 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.653210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:35976)
0812 07:01:58.653889 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.654053 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":310,"threads_started":1}
W20260812 07:01:58.654493 32680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:33271: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
I20260812 07:01:58.722782 32680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } attempt: 2
I20260812 07:01:58.724036 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.723474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:35978)
0812 07:01:58.723734 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.723860 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 07:01:58.725219 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.724777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:35994)
0812 07:01:58.724950 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.725063 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 07:01:58.725467 32680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:33271: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
I20260812 07:01:58.816787 32680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } attempt: 3
I20260812 07:01:58.818102 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.817648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:35998)
0812 07:01:58.817861 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.817959 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 07:01:58.819288 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.818897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:36004)
0812 07:01:58.819070 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.819143 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 07:01:58.819540 32680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:33271: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
I20260812 07:01:58.962785 32680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } attempt: 4
I20260812 07:01:58.964226 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.963580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:36014)
0812 07:01:58.963925 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.964019 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260812 07:01:58.965294 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:58.964931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:36022)
0812 07:01:58.965096 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:58.965166 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 07:01:58.965487 32680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:33271: Network error: Client connection negotiation failed: client connection to 127.26.226.61:33271: connect: Connection refused (error 111)
W20260812 07:01:59.015918 32689 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:01:59.016461 32689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:59.016629 32689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:59.016788 32689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:59.025616 32689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:59.025763 32689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:59.025902 32689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:59.026000 32689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:59.026127 32689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:59.038244 32689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:43265,127.26.226.61:33271,127.26.226.60:32847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab
--principal=kudu/127.26.226.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.61:33271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.226.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:59.039785 32689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:59.041319 32689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:59.054457 32698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:59.054617 32701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:59.054828 32697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:59.055450 32689 server_base.cc:1034] running on GCE node
I20260812 07:01:59.056356 32689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:59.057868 32689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:59.059126 32689 hybrid_clock.cc:648] HybridClock initialized: now 1786518119059061 us; error 43 us; skew 500 ppm
I20260812 07:01:59.059495 32689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:59.062834 32689 init.cc:377] Logged in from keytab as kudu/127.26.226.61@KRBTEST.COM (short username kudu)
I20260812 07:01:59.064436 32689 webserver.cc:460] Webserver started at http://127.26.226.61:41431/ using document root <none> and password file <none>
I20260812 07:01:59.065160 32689 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:59.065325 32689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:59.065657 32689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:59.068778 32689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/data/instance:
uuid: "b8a5c792f5b741daa39866cf3f0fff8f"
format_stamp: "Formatted at 2026-08-12 07:01:59 on dist-test-slave-t3q3"
server_key: "8834b0ae7038fbcfef8591dc29974312"
server_key_iv: "979e17e9f6631564bd570314532c8fe6"
server_key_version: "encryptionkey@0"
I20260812 07:01:59.069638 32689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/wal/instance:
uuid: "b8a5c792f5b741daa39866cf3f0fff8f"
format_stamp: "Formatted at 2026-08-12 07:01:59 on dist-test-slave-t3q3"
server_key: "8834b0ae7038fbcfef8591dc29974312"
server_key_iv: "979e17e9f6631564bd570314532c8fe6"
server_key_version: "encryptionkey@0"
I20260812 07:01:59.074857 32689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 07:01:59.078626 32713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:59.080147 32689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:01:59.080361 32689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/wal
uuid: "b8a5c792f5b741daa39866cf3f0fff8f"
format_stamp: "Formatted at 2026-08-12 07:01:59 on dist-test-slave-t3q3"
server_key: "8834b0ae7038fbcfef8591dc29974312"
server_key_iv: "979e17e9f6631564bd570314532c8fe6"
server_key_version: "encryptionkey@0"
I20260812 07:01:59.080614 32689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:59.094787 32689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:59.099152 32689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:59.099452 32689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:59.118775 32689 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.61:33271
I20260812 07:01:59.118798 329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.61:33271 every 8 connection(s)
I20260812 07:01:59.120289 32689 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
I20260812 07:01:59.124049 330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:59.126749 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 32689
I20260812 07:01:59.126973 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-1/wal/instance
I20260812 07:01:59.127444 27528 external_mini_cluster.cc:1428] Setting key a21e9a845a12d1e5c5afbbf603bd6938
I20260812 07:01:59.131001 330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:59.153179 333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.133881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37374 (local address 127.26.226.62:43265)
0812 07:01:59.134423 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.134428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.135497 (+ 1069us) server_negotiation.cc:407] Connection header received
0812 07:01:59.137575 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.137580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.137683 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.137806 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.139410 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.140288 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.141829 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.142263 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.142404 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.146680 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.146686 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.146690 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.146736 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.149761 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.150587 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.150600 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.150602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.150737 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.151201 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.151208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.151216 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.151512 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.151696 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.152508 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.152749 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":305,"threads_started":1}
I20260812 07:01:59.154485 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.134537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43265 (local address 127.0.0.1:37374)
0812 07:01:59.135326 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.135364 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:59.137270 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:59.138076 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:59.138084 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:59.138630 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:59.139222 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.139240 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.140457 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.140462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.141660 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.141673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.142502 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.142509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.142764 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.143495 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:59.143519 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:59.146397 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:59.149934 (+ 3537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.149944 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.149963 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.150443 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.150857 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.150862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.150865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.151073 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.151662 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:59.151679 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:59.152054 (+ 375us) client_negotiation.cc:769] Sending connection context
0812 07:01:59.152422 (+ 368us) client_negotiation.cc:240] Negotiation successful
0812 07:01:59.153021 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":344,"threads_started":1}
I20260812 07:01:59.156548 330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.226.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.60@KRBTEST.COM" created.
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 07:01:59.175869 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.157539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36038 (local address 127.26.226.61:33271)
0812 07:01:59.158349 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.158360 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.158395 (+ 35us) server_negotiation.cc:407] Connection header received
0812 07:01:59.158457 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.158467 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.158722 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.158889 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.159613 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.161526 (+ 1913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.162949 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.163959 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.164178 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.169308 (+ 5130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.169323 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.169339 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.169371 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.172662 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.173598 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.173606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.173609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.173747 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.174185 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.174188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.174190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.174562 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.174748 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.175166 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.175511 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":179,"server-negotiator.queue_time_us":601,"thread_start_us":431,"threads_started":1}
I20260812 07:01:59.175977 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.157297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:36038)
0812 07:01:59.157524 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.157543 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:59.157659 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:59.158891 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:59.158895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:59.159283 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:59.159429 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.159437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.161680 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.161684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.162791 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.162804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.164176 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.164180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.164380 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.165054 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:59.165073 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:59.169028 (+ 3955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:59.172836 (+ 3808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.172844 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.172851 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.173442 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.173867 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.173875 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.173881 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.174072 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.174696 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:59.174704 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:59.174873 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 07:01:59.175034 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 07:01:59.175392 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":38}
I20260812 07:01:59.178333 330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:01:59.179742 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.179150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:45236)
0812 07:01:59.179375 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.179517 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 07:01:59.180042 32723 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
I20260812 07:01:59.181799 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.181352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:45246)
0812 07:01:59.181537 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.181650 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 07:01:59.182014 330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:32847: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab.
Entry for principal kudu/127.26.226.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab.
I20260812 07:01:59.210842 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.60:32847
--webserver_interface=127.26.226.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab
--principal=kudu/127.26.226.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:43265,127.26.226.61:33271,127.26.226.60:32847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:01:59.231359 330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } attempt: 1
I20260812 07:01:59.232638 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.232176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:45254)
0812 07:01:59.232374 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.232482 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 07:01:59.233716 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.233380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:45260)
0812 07:01:59.233529 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.233604 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:01:59.234011 330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:32847: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
I20260812 07:01:59.264745 32680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } attempt: 5
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518118, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 07:01:59.278330 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.265662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36050 (local address 127.26.226.61:33271)
0812 07:01:59.265858 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.265866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.265891 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:01:59.266141 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.266146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.266215 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.266327 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.267135 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.267857 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.269086 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.269549 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.269694 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.272907 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.272912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.272916 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.272948 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.275632 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.276406 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.276418 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.276422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.276541 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.276930 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.276935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.276945 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.277227 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.277382 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.277788 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.278006 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:01:59.278373 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.265567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:36050)
0812 07:01:59.265796 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.265818 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:59.265972 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:59.266363 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:59.266368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:59.266805 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:59.266973 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.266983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.267996 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.268001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.268932 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.268944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.269710 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.269716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.269889 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.270542 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:59.270560 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:59.272705 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:59.275787 (+ 3082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.275793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.275797 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.276280 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.276658 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.276663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.276667 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.276809 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.277350 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:59.277355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:59.277515 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 07:01:59.277652 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 07:01:59.277941 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 07:01:59.279724 32680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:01:59.280793 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.280403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:45272)
0812 07:01:59.280571 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.280659 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 07:01:59.281796 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.281464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:45276)
0812 07:01:59.281625 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.281700 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 07:01:59.282006 32680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:32847: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
I20260812 07:01:59.302256 330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } attempt: 2
I20260812 07:01:59.303478 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.303017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:45292)
0812 07:01:59.303229 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.303321 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 07:01:59.304761 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.304317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:45294)
0812 07:01:59.304497 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.304591 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 07:01:59.305078 330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:32847: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
I20260812 07:01:59.333253 32680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } attempt: 1
I20260812 07:01:59.334594 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.334043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:45298)
0812 07:01:59.334312 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.334410 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 07:01:59.335759 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.335391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:45312)
0812 07:01:59.335566 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.335639 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 07:01:59.335979 32680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:32847: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
I20260812 07:01:59.396355 330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } attempt: 3
I20260812 07:01:59.397710 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.397205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55092)
0812 07:01:59.397430 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.397536 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 07:01:59.398814 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.398381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55102)
0812 07:01:59.398613 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.398697 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260812 07:01:59.399036 330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:32847: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
I20260812 07:01:59.404217 32680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } attempt: 2
I20260812 07:01:59.405259 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.404803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55106)
0812 07:01:59.405007 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.405104 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 07:01:59.406313 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.405923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55122)
0812 07:01:59.406101 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.406182 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 07:01:59.406520 32680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:32847: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
I20260812 07:01:59.512763 32680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } attempt: 3
I20260812 07:01:59.513937 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.513477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55126)
0812 07:01:59.513701 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.513789 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 07:01:59.514998 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.514645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55138)
0812 07:01:59.514829 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.514892 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 07:01:59.515208 32680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:32847: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
I20260812 07:01:59.542273 330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } attempt: 4
I20260812 07:01:59.543463 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.542976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55150)
0812 07:01:59.543219 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.543307 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 07:01:59.544476 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.544154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55152)
0812 07:01:59.544296 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.544371 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 07:01:59.544677 330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:32847: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
W20260812 07:01:59.582751 339 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:01:59.583187 339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:01:59.583352 339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:01:59.583503 339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:01:59.591131 339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:01:59.591265 339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:01:59.591401 339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:01:59.591499 339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:01:59.591642 339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:01:59.602246 339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:43265,127.26.226.61:33271,127.26.226.60:32847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab
--principal=kudu/127.26.226.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.60:32847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.226.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:01:59.603679 339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:01:59.605099 339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:01:59.618104 348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:59.618217 354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:01:59.618266 349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:01:59.619064 339 server_base.cc:1034] running on GCE node
I20260812 07:01:59.619869 339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:01:59.621413 339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:01:59.622666 339 hybrid_clock.cc:648] HybridClock initialized: now 1786518119622556 us; error 79 us; skew 500 ppm
I20260812 07:01:59.623008 339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:01:59.626291 339 init.cc:377] Logged in from keytab as kudu/127.26.226.60@KRBTEST.COM (short username kudu)
I20260812 07:01:59.628010 339 webserver.cc:460] Webserver started at http://127.26.226.60:33087/ using document root <none> and password file <none>
I20260812 07:01:59.628847 339 fs_manager.cc:359] Metadata directory not provided
I20260812 07:01:59.629060 339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:01:59.629439 339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:01:59.632604 339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/data/instance:
uuid: "c9908eda313449269f74279e39dd9198"
format_stamp: "Formatted at 2026-08-12 07:01:59 on dist-test-slave-t3q3"
server_key: "2df14ce1c40d0e145adbc68c15c4a4c6"
server_key_iv: "ec82fea2dd351e7561338ff2b36ff0ac"
server_key_version: "encryptionkey@0"
I20260812 07:01:59.633446 339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/wal/instance:
uuid: "c9908eda313449269f74279e39dd9198"
format_stamp: "Formatted at 2026-08-12 07:01:59 on dist-test-slave-t3q3"
server_key: "2df14ce1c40d0e145adbc68c15c4a4c6"
server_key_iv: "ec82fea2dd351e7561338ff2b36ff0ac"
server_key_version: "encryptionkey@0"
I20260812 07:01:59.638957 339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 07:01:59.642863 365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:01:59.644546 339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:01:59.644762 339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/wal
uuid: "c9908eda313449269f74279e39dd9198"
format_stamp: "Formatted at 2026-08-12 07:01:59 on dist-test-slave-t3q3"
server_key: "2df14ce1c40d0e145adbc68c15c4a4c6"
server_key_iv: "ec82fea2dd351e7561338ff2b36ff0ac"
server_key_version: "encryptionkey@0"
I20260812 07:01:59.645022 339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:01:59.671341 339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:01:59.675737 339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:01:59.676144 339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:01:59.692428 32680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } attempt: 4
I20260812 07:01:59.693789 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.693248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55162)
0812 07:01:59.693448 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.693573 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 07:01:59.694962 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.694572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55168)
0812 07:01:59.694735 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.694834 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 07:01:59.695271 32680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:32847: Network error: Client connection negotiation failed: client connection to 127.26.226.60:32847: connect: Connection refused (error 111)
I20260812 07:01:59.696246 339 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.60:32847
I20260812 07:01:59.696269 458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.60:32847 every 8 connection(s)
I20260812 07:01:59.697727 339 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
I20260812 07:01:59.699625 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 339
I20260812 07:01:59.699829 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517734245872-27528-0/minicluster-data/master-2/wal/instance
I20260812 07:01:59.700430 27528 external_mini_cluster.cc:1428] Setting key 07db66cbee27243e70f1eca63fee8eec
I20260812 07:01:59.702550 459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:01:59.711845 459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:01:59.726364 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.702427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.26.226.60:32847)
0812 07:01:59.703367 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.703379 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.703421 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:01:59.705206 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.705230 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.705671 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.706045 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.708230 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.709697 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.711537 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.712335 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.712497 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.717987 (+ 5490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.717995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.718006 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.718050 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.721421 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.722360 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.722367 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.722374 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.722613 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.723126 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.723132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.723135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.723728 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.723971 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.724581 (+ 610us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.725087 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":508,"threads_started":1}
I20260812 07:01:59.732012 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.715377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49666 (local address 127.26.226.62:43265)
0812 07:01:59.716096 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.716103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.716613 (+ 510us) server_negotiation.cc:407] Connection header received
0812 07:01:59.716892 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.716897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.716964 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.717061 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.718581 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.719824 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.721498 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.722003 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.722136 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.725740 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.725747 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.725752 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.725791 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.728971 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.729807 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.729815 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.729820 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.730021 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.730442 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.730446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.730449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.730787 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.730920 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.731493 (+ 573us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.731657 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":298,"threads_started":1}
I20260812 07:01:59.732189 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.715354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:43265 (local address 127.0.0.1:49666)
0812 07:01:59.716444 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.716495 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:59.716715 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:59.717189 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:59.717200 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:59.718077 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:59.718413 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.718423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.720020 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.720024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.721364 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.721377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.722194 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.722200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.722403 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.723033 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:59.723070 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:59.725536 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:59.729087 (+ 3551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.729092 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.729097 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.729644 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.730164 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.730168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.730191 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.730324 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.730927 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:59.730935 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:59.731154 (+ 219us) client_negotiation.cc:769] Sending connection context
0812 07:01:59.731383 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 07:01:59.731699 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":613,"threads_started":1}
I20260812 07:01:59.733637 459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 07:01:59.746168 466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.729638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49674 (local address 127.26.226.62:43265)
0812 07:01:59.730220 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.730225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.730237 (+ 12us) server_negotiation.cc:407] Connection header received
0812 07:01:59.730272 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.730275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.730327 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.730443 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.731969 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.733227 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.734811 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.735344 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.735438 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.740454 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.740464 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.740472 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.740515 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.743217 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.744096 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.744103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.744112 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.744240 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.744653 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.744656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.744658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.744925 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.745166 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.745475 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.745711 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":452,"thread_start_us":293,"threads_started":1}
I20260812 07:01:59.747977 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.734280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:33271 (local address 127.0.0.1:40924)
0812 07:01:59.734503 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.734520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:59.734612 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:59.734958 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:59.734961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:59.735288 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:59.735461 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.735472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.736600 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.736607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.738117 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.738138 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.738837 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.738846 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.739021 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.739596 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:59.739612 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:59.741753 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:59.745135 (+ 3382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.745140 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.745144 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.745650 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.746107 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.746112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.746118 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.746309 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.747009 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:59.747014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:59.747176 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 07:01:59.747319 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 07:01:59.747605 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260812 07:01:59.748029 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.734390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.26.226.61:33271)
0812 07:01:59.734555 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.734560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.734576 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:01:59.734724 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.734728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.734822 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.734924 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.735630 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.736453 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.738278 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.738672 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.738764 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.741949 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.741956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.741961 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.742002 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.744994 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.745792 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.745797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.745801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.745955 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.746512 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.746518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.746520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.746882 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.747042 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.747409 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.747618 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:01:59.748968 459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 07:01:59.763801 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.750183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.26.226.60:32847)
0812 07:01:59.750337 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.750342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.750360 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:01:59.750404 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.750408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.750477 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.750582 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.751433 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.752356 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.753656 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.754215 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.754356 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.758112 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.758116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.758119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.758148 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.760944 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.761749 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.761753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.761755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.761898 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.762305 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.762313 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.762316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.762564 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.762735 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.763068 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.763322 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":53}
I20260812 07:01:59.763825 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.749848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55198)
0812 07:01:59.750040 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.750060 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:59.750174 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:59.750602 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:59.750606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:59.751112 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:59.751291 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.751303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.752480 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.752484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.753506 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.753521 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.754403 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.754411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.754718 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.755283 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:59.755307 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:59.757910 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:59.761089 (+ 3179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.761094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.761098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.761625 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.762037 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.762042 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.762046 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.762197 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.762701 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:59.762706 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:59.762840 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 07:01:59.762989 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 07:01:59.763472 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":174}
I20260812 07:01:59.764886 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.748315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40940 (local address 127.26.226.61:33271)
0812 07:01:59.748493 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.748499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.748680 (+ 181us) server_negotiation.cc:407] Connection header received
0812 07:01:59.748828 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.748833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.748900 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.749027 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.750681 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.751677 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.753131 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.753751 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.753840 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.758899 (+ 5059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.758904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.758907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.758936 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.761840 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.762483 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.762488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.762498 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.762624 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.763029 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.763033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.763035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.763271 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.763551 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.764433 (+ 882us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.764663 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:01:59.769944 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:01:59.770074 459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198: Bootstrap starting.
I20260812 07:01:59.773617 459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:59.774950 459 log.cc:824] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:59.772666 32642 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7f2da853df8e43c685765a00db43339b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49674
I20260812 07:01:59.778151 459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198: No bootstrap required, opened a new log
I20260812 07:01:59.788944 459 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } }
I20260812 07:01:59.789391 459 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9908eda313449269f74279e39dd9198, State: Initialized, Role: FOLLOWER
I20260812 07:01:59.789889 459 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } }
I20260812 07:01:59.790828 468 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } } }
I20260812 07:01:59.791200 468 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:01:59.792104 459 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:01:59.802011 480 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:01:59.802151 480 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:01:59.799772 339 master_runner.cc:186] Error getting master registration for 127.26.226.62:43265: OK, instance_id { permanent_uuid: "7f2da853df8e43c685765a00db43339b" instance_seqno: 1786518118553202 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:01:59.811237 339 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:01:59.843905 330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } attempt: 5
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
I20260812 07:01:59.860841 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.844772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55200)
0812 07:01:59.845003 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.845027 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:59.845168 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:59.845580 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:59.845586 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:59.846019 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:59.846188 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.846198 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.847247 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.847256 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.848217 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.848230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.849303 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.849312 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.849607 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.850361 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:59.850391 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:59.853449 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:59.857832 (+ 4383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.857840 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.857846 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.858397 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.858849 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.858853 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.858857 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.859027 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.859764 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:59.859769 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:59.859923 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 07:01:59.860117 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 07:01:59.860433 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 07:01:59.860857 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.844851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.26.226.60:32847)
0812 07:01:59.845138 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.845154 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.845179 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:01:59.845334 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.845339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.845431 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.845532 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.846361 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.847090 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.848366 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.849089 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.849233 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.853680 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.853688 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.853695 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.853736 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.857670 (+ 3934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.858535 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.858541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.858545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.858725 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.859146 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.859152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.859155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.859637 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.859819 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.860237 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.860476 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:01:59.865716 330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f: Bootstrap starting.
I20260812 07:01:59.868898 330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f: Neither blocks nor log segments found. Creating new log.
I20260812 07:01:59.870009 330 log.cc:824] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:01:59.872995 330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f: No bootstrap required, opened a new log
I20260812 07:01:59.884008 330 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } }
I20260812 07:01:59.884465 330 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8a5c792f5b741daa39866cf3f0fff8f, State: Initialized, Role: FOLLOWER
I20260812 07:01:59.884966 330 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } }
I20260812 07:01:59.886081 490 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } } }
I20260812 07:01:59.886512 490 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:01:59.887369 330 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:01:59.898082 504 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:01:59.898355 504 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:01:59.895452 32689 master_runner.cc:186] Error getting master registration for 127.26.226.62:43265: OK, instance_id { permanent_uuid: "7f2da853df8e43c685765a00db43339b" instance_seqno: 1786518118553202 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:01:59.907213 32689 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 07:01:59.928349 374 tablet.cc:2369] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:01:59.972548 32680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } attempt: 5
Aug 12 07:01:59 dist-test-slave-t3q3 krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518118, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
I20260812 07:01:59.992002 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.973382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:32847 (local address 127.0.0.1:55216)
0812 07:01:59.973611 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 07:01:59.973644 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 07:01:59.973788 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:01:59.974283 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:01:59.974289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:01:59.974859 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:01:59.975073 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.975087 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.976724 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.976730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.978252 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:01:59.978269 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.979673 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.979681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:01:59.979908 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.980701 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:01:59.980727 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:01:59.983780 (+ 3053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:01:59.988697 (+ 4917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.988704 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.988708 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.989330 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.989934 (+ 604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:01:59.989939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:01:59.989943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:01:59.990169 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:01:59.990852 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:01:59.990858 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:01:59.991055 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 07:01:59.991244 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 07:01:59.991571 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 07:01:59.993104 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:01:59.973716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55216 (local address 127.26.226.60:32847)
0812 07:01:59.973926 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:01:59.973932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:01:59.973954 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:01:59.974029 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:01:59.974034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:01:59.974116 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:01:59.974242 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:01:59.975293 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.976569 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.978477 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:01:59.979266 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:01:59.979417 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:01:59.984041 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:01:59.984048 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:01:59.984053 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:01:59.984125 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:01:59.988481 (+ 4356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.989553 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.989559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.989563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.989780 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:01:59.990298 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:01:59.990303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:01:59.990307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:01:59.990711 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:01:59.990873 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:01:59.992454 (+ 1581us) server_negotiation.cc:299] Negotiation successful
0812 07:01:59.992714 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:01:59.999445 32680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b: Bootstrap starting.
I20260812 07:02:00.003895 32680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:00.005398 32680 log.cc:824] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:00.009514 32680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b: No bootstrap required, opened a new log
I20260812 07:02:00.018044 32680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } }
I20260812 07:02:00.018489 32680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f2da853df8e43c685765a00db43339b, State: Initialized, Role: FOLLOWER
I20260812 07:02:00.019021 32680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } }
I20260812 07:02:00.020349 511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } } }
I20260812 07:02:00.021047 511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:00.021618 32680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:00.021859 32642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 07:02:00.022065 32642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:00.025535 32642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } }
I20260812 07:02:00.028146 32755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f2da853df8e43c685765a00db43339b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b8a5c792f5b741daa39866cf3f0fff8f"
I20260812 07:02:00.028692 32755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:00.032258 32755 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f2da853df8e43c685765a00db43339b in term 1.
I20260812 07:02:00.033119 32600 leader_election.cc:302] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f2da853df8e43c685765a00db43339b, b8a5c792f5b741daa39866cf3f0fff8f; no voters:
I20260812 07:02:00.034718 422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f2da853df8e43c685765a00db43339b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c9908eda313449269f74279e39dd9198"
I20260812 07:02:00.035220 422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:00.039078 422 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f2da853df8e43c685765a00db43339b in term 1.
I20260812 07:02:00.039825 32642 leader_election.cc:288] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [CANDIDATE]: Term 1 election: Requested vote from peers b8a5c792f5b741daa39866cf3f0fff8f (127.26.226.61:33271), c9908eda313449269f74279e39dd9198 (127.26.226.60:32847)
I20260812 07:02:00.040970 511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [term 1 FOLLOWER]: Leader election won for term 1
W20260812 07:02:00.042943 32605 tablet.cc:2369] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:00.072921 511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [term 1 LEADER]: Becoming Leader. State: Replica: 7f2da853df8e43c685765a00db43339b, State: Running, Role: LEADER
I20260812 07:02:00.073822 511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } }
I20260812 07:02:00.084980 518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2da853df8e43c685765a00db43339b. Latest consensus state: current_term: 1 leader_uuid: "7f2da853df8e43c685765a00db43339b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } } }
I20260812 07:02:00.085579 518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:00.088897 540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:00.100769 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:00.073642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55220 (local address 127.26.226.60:32847)
0812 07:02:00.073899 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.073906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.077581 (+ 3675us) server_negotiation.cc:407] Connection header received
0812 07:02:00.077838 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.077843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.077931 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.078016 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.081237 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:00.082287 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:00.084793 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:00.085603 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:00.085736 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:00.092166 (+ 6430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:00.092176 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:00.092184 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:00.092229 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:00.096608 (+ 4379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:00.097603 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:00.097609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:00.097613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:00.097778 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:00.098230 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:00.098235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:00.098239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:00.098618 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:00.098895 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:00.100091 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0812 07:02:00.100368 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 07:02:00.102334 32725 tablet.cc:2369] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:00.111690 540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:00.118870 549 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:02:00.124811 422 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [term 1 FOLLOWER]: Refusing update from remote peer 7f2da853df8e43c685765a00db43339b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:00.124920 32755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [term 1 FOLLOWER]: Refusing update from remote peer 7f2da853df8e43c685765a00db43339b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:00.126223 518 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:00.127259 518 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:00.131855 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:00.078551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40956 (local address 127.26.226.61:33271)
0812 07:02:00.078774 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.078781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.090028 (+ 11247us) server_negotiation.cc:407] Connection header received
0812 07:02:00.101542 (+ 11514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.101549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.101668 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.101751 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.104167 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:00.105261 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:00.108375 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:00.109071 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:00.109179 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:00.115841 (+ 6662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:00.115847 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:00.115851 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:00.115901 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:00.119955 (+ 4054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:00.126401 (+ 6446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:00.126407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:00.126410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:00.126606 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:00.127786 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:00.127791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:00.127794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:00.128261 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:00.128512 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:00.131034 (+ 2522us) server_negotiation.cc:299] Negotiation successful
0812 07:02:00.131458 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:02:00.133252 466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:00.071831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.26.226.62:43265)
0812 07:02:00.072112 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.072120 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.077568 (+ 5448us) server_negotiation.cc:407] Connection header received
0812 07:02:00.077789 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.077794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.077883 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.077999 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.084904 (+ 6905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:00.086070 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:00.108343 (+ 22273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:00.109076 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:00.109234 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:00.120963 (+ 11729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:00.120971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:00.120975 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:00.121023 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:00.125095 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:00.127121 (+ 2026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:00.127127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:00.127130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:00.127291 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:00.129026 (+ 1735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:00.129031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:00.129034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:00.130016 (+ 982us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:00.130288 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:00.132678 (+ 2390us) server_negotiation.cc:299] Negotiation successful
0812 07:02:00.132931 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:02:00.153728 490 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2da853df8e43c685765a00db43339b. Latest consensus state: current_term: 1 leader_uuid: "7f2da853df8e43c685765a00db43339b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } } }
I20260812 07:02:00.154114 490 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:00.166433 550 mvcc.cc:205] Tried to move back new op lower bound from 7317578220016267264 to 7317578219829866496. Current Snapshot: MvccSnapshot[applied={T|T < 7317578220016267264}]
I20260812 07:02:00.168942 490 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f2da853df8e43c685765a00db43339b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } } }
I20260812 07:02:00.169266 490 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:00.170413 551 mvcc.cc:205] Tried to move back new op lower bound from 7317578220016267264 to 7317578219829866496. Current Snapshot: MvccSnapshot[applied={T|T < 7317578220016267264 or (T in {7317578220016267264})}]
I20260812 07:02:00.172610 518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f2da853df8e43c685765a00db43339b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } } }
I20260812 07:02:00.172897 518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:00.174054 540 catalog_manager.cc:1347] Generated new cluster ID: 1e86aaef9af7476c970785a8ae3edc72
I20260812 07:02:00.174171 540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:00.198654 468 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2da853df8e43c685765a00db43339b. Latest consensus state: current_term: 1 leader_uuid: "7f2da853df8e43c685765a00db43339b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } } }
I20260812 07:02:00.199576 468 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:00.202021 511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f2da853df8e43c685765a00db43339b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } } }
I20260812 07:02:00.202360 511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:00.207604 540 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:00.209599 540 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:00.226958 468 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f2da853df8e43c685765a00db43339b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2da853df8e43c685765a00db43339b" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 43265 } } peers { permanent_uuid: "b8a5c792f5b741daa39866cf3f0fff8f" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 33271 } } peers { permanent_uuid: "c9908eda313449269f74279e39dd9198" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 32847 } } }
I20260812 07:02:00.227277 468 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:00.236044 540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f2da853df8e43c685765a00db43339b: Generated new TSK 0
I20260812 07:02:00.237972 540 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:02:00.403505 466 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.382774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.26.226.62:43265)
0812 07:02:00.383009 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.383016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.397927 (+ 14911us) server_negotiation.cc:407] Connection header received
0812 07:02:00.401348 (+ 3421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.401356 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.401505 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.401608 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.403134 (+ 1526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49692: BlockingRecv error: recv got EOF from 127.0.0.1:49692 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260812 07:02:00.410326 462 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.391399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55228 (local address 127.26.226.60:32847)
0812 07:02:00.391649 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.391657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.406276 (+ 14619us) server_negotiation.cc:407] Connection header received
0812 07:02:00.408459 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.408465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.408725 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.408841 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.410029 (+ 1188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55228: BlockingRecv error: recv got EOF from 127.0.0.1:55228 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260812 07:02:00.412865 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.398634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.26.226.61:33271)
0812 07:02:00.398874 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.398882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.407662 (+ 8780us) server_negotiation.cc:407] Connection header received
0812 07:02:00.411010 (+ 3348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.411017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.411182 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.411266 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.412608 (+ 1342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40966: BlockingRecv error: recv got EOF from 127.0.0.1:40966 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260812 07:02:00.449064 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.446639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.26.226.61:33271)
0812 07:02:00.446848 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.446855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.446882 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:02:00.447221 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.447228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.447395 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.447474 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.448719 (+ 1245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40972: BlockingRecv error: recv got EOF from 127.0.0.1:40972 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260812 07:02:00.459231 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.456887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.26.226.61:33271)
0812 07:02:00.457071 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.457083 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.457660 (+ 577us) server_negotiation.cc:407] Connection header received
0812 07:02:00.457866 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.457872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.457967 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.458076 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.458987 (+ 911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40986: BlockingRecv error: recv got EOF from 127.0.0.1:40986 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 07:02:00.466920 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.464660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40992 (local address 127.26.226.61:33271)
0812 07:02:00.464848 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.464854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.465354 (+ 500us) server_negotiation.cc:407] Connection header received
0812 07:02:00.465568 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.465573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.465688 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.465797 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.466637 (+ 840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40992: BlockingRecv error: recv got EOF from 127.0.0.1:40992 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 07:02:00.476743 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.474465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.26.226.61:33271)
0812 07:02:00.474655 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.474661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.475199 (+ 538us) server_negotiation.cc:407] Connection header received
0812 07:02:00.475385 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.475391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.475509 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.475620 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.476483 (+ 863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41008: BlockingRecv error: recv got EOF from 127.0.0.1:41008 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 07:02:00.500685 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.498273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41014 (local address 127.26.226.61:33271)
0812 07:02:00.498481 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.498489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.499058 (+ 569us) server_negotiation.cc:407] Connection header received
0812 07:02:00.499273 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.499281 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.499381 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.499507 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.500430 (+ 923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41014: BlockingRecv error: recv got EOF from 127.0.0.1:41014 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 07:02:00.523041 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.520662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41028 (local address 127.26.226.61:33271)
0812 07:02:00.520849 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.520857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.521393 (+ 536us) server_negotiation.cc:407] Connection header received
0812 07:02:00.521707 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.521713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.521815 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.521935 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.522783 (+ 848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41028: BlockingRecv error: recv got EOF from 127.0.0.1:41028 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 07:02:00.563048 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.560487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.26.226.61:33271)
0812 07:02:00.560697 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.560704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.561427 (+ 723us) server_negotiation.cc:407] Connection header received
0812 07:02:00.561641 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.561647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.561738 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.561842 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.562784 (+ 942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41032: BlockingRecv error: recv got EOF from 127.0.0.1:41032 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260812 07:02:00.633276 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.631279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41048 (local address 127.26.226.61:33271)
0812 07:02:00.631524 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.631533 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.631825 (+ 292us) server_negotiation.cc:407] Connection header received
0812 07:02:00.631962 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.631967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.632102 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.632250 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.633035 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41048: BlockingRecv error: recv got EOF from 127.0.0.1:41048 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260812 07:02:00.767359 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:00.765450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.26.226.61:33271)
0812 07:02:00.765745 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:00.765752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:00.765954 (+ 202us) server_negotiation.cc:407] Connection header received
0812 07:02:00.766126 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:00.766132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:00.766232 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:00.766356 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:00.767085 (+ 729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41056: BlockingRecv error: recv got EOF from 127.0.0.1:41056 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
I20260812 07:02:00.804296 480 catalog_manager.cc:1259] Loaded cluster ID: 1e86aaef9af7476c970785a8ae3edc72
I20260812 07:02:00.804442 480 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198: loading cluster ID for follower catalog manager: success
I20260812 07:02:00.806988 480 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198: acquiring CA information for follower catalog manager: success
I20260812 07:02:00.808830 480 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c9908eda313449269f74279e39dd9198: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:02:00.900524 504 catalog_manager.cc:1259] Loaded cluster ID: 1e86aaef9af7476c970785a8ae3edc72
I20260812 07:02:00.900631 504 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f: loading cluster ID for follower catalog manager: success
I20260812 07:02:00.903028 504 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f: acquiring CA information for follower catalog manager: success
I20260812 07:02:00.905105 504 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b8a5c792f5b741daa39866cf3f0fff8f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:02:01.036849 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:01.031328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.26.226.61:33271)
0812 07:02:01.031511 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:01.031515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:01.031823 (+ 308us) server_negotiation.cc:407] Connection header received
0812 07:02:01.032006 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:01.032010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:01.032245 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:01.032360 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:01.032773 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:01.033510 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:01.034822 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:01.035196 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:01.035292 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:01.035752 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:01.036330 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:01.036440 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:01.036541 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 07:02:01.036608 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:02:01.049311 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:01.043189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.26.226.60:32847)
0812 07:02:01.043751 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:01.043758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:01.044003 (+ 245us) server_negotiation.cc:407] Connection header received
0812 07:02:01.044281 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:01.044286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:01.044448 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:01.044594 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:01.045012 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:01.045743 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:01.046998 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:01.047386 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:01.047475 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:01.047941 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:01.048567 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:01.048723 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:01.048830 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 07:02:01.048951 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":289,"threads_started":1}
I20260812 07:02:01.064121 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:01.055999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.26.226.60:32847)
0812 07:02:01.056262 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:01.056271 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:01.058501 (+ 2230us) server_negotiation.cc:407] Connection header received
0812 07:02:01.058769 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:01.058774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:01.058857 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:01.058920 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:01.059501 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:01.060509 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:01.062181 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:01.062715 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:01.062843 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:01.063282 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:01.063409 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:01.063504 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:01.063696 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 07:02:01.063778 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 07:02:01.064322 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:01.055832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49704 (local address 127.26.226.62:43265)
0812 07:02:01.057411 (+ 1579us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:01.057423 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:01.057457 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:02:01.057555 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:01.057558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:01.057708 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:01.057854 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:01.058603 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:01.059949 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:01.061748 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:01.062202 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:01.062330 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:01.062871 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:01.063296 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:01.063418 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:01.063881 (+ 463us) server_negotiation.cc:299] Negotiation successful
0812 07:02:01.063973 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1443,"thread_start_us":1339,"threads_started":1}
I20260812 07:02:01.067890 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:01.058622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.26.226.61:33271)
0812 07:02:01.058820 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:01.058825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:01.060048 (+ 1223us) server_negotiation.cc:407] Connection header received
0812 07:02:01.061363 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:01.061369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:01.061459 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:01.061561 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:01.062281 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:01.063347 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:01.065840 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:01.066465 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:01.066595 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:01.067098 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:01.067248 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:01.067323 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:01.067495 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 07:02:01.067558 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 07:02:01.070369 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32575
I20260812 07:02:01.071425 32672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:01.182412 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 32575
W20260812 07:02:01.216239 32716 connection.cc:535] server connection from 127.0.0.1:36050 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 07:02:01.216640 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 32689
I20260812 07:02:01.217833 320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:01.330008 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 32689
I20260812 07:02:01.363076 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 339
I20260812 07:02:01.364295 453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:01.474068 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 339
2026-08-12T07:02:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5190 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:02:01 dist-test-slave-t3q3 krb5kdc[689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:02:01 dist-test-slave-t3q3 krb5kdc[689](info): set up 2 sockets
Aug 12 07:02:01 dist-test-slave-t3q3 krb5kdc[689](info): commencing operation
krb5kdc: starting...
W20260812 07:02:02.900478 27528 mini_kdc.cc:121] Time spent starting KDC: real 1.371s user 0.002s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/test-user.keytab.
Aug 12 07:02:03 dist-test-slave-t3q3 krb5kdc[689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:02:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.226.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
Entry for principal kudu/127.26.226.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab.
I20260812 07:02:03.283152 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:42349
--webserver_interface=127.26.226.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:42349,127.26.226.61:35037,127.26.226.60:45005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:03.656868 705 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:02:03.657344 705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:03.657473 705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:03.657634 705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:03.665637 705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:03.665802 705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:03.665920 705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:03.666024 705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:03.666126 705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:03.676674 705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:42349,127.26.226.61:35037,127.26.226.60:45005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.62.keytab
--principal=kudu/127.26.226.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.62:42349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.226.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:03.678195 705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:03.679646 705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:03.692981 711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:03.693007 714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:03.692981 712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:03.694670 705 server_base.cc:1034] running on GCE node
I20260812 07:02:03.695542 705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:03.697161 705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:03.698424 705 hybrid_clock.cc:648] HybridClock initialized: now 1786518123698356 us; error 30 us; skew 500 ppm
I20260812 07:02:03.698870 705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:03 dist-test-slave-t3q3 krb5kdc[689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518123, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:03.712411 705 init.cc:377] Logged in from keytab as kudu/127.26.226.62@KRBTEST.COM (short username kudu)
I20260812 07:02:03.714766 705 webserver.cc:460] Webserver started at http://127.26.226.62:41045/ using document root <none> and password file <none>
I20260812 07:02:03.715924 705 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:03.716105 705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:03.716621 705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:03.721330 705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/data/instance:
uuid: "ce0533c54e50486aa1584ca023251f33"
format_stamp: "Formatted at 2026-08-12 07:02:03 on dist-test-slave-t3q3"
server_key: "cc2f3e27497b77ac23777ba402509765"
server_key_iv: "75637fee061b2a0c0b57bc1ae6791cda"
server_key_version: "encryptionkey@0"
I20260812 07:02:03.722599 705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/wal/instance:
uuid: "ce0533c54e50486aa1584ca023251f33"
format_stamp: "Formatted at 2026-08-12 07:02:03 on dist-test-slave-t3q3"
server_key: "cc2f3e27497b77ac23777ba402509765"
server_key_iv: "75637fee061b2a0c0b57bc1ae6791cda"
server_key_version: "encryptionkey@0"
I20260812 07:02:03.730660 705 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.001s
I20260812 07:02:03.735672 728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:03.737833 705 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:02:03.738122 705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/wal
uuid: "ce0533c54e50486aa1584ca023251f33"
format_stamp: "Formatted at 2026-08-12 07:02:03 on dist-test-slave-t3q3"
server_key: "cc2f3e27497b77ac23777ba402509765"
server_key_iv: "75637fee061b2a0c0b57bc1ae6791cda"
server_key_version: "encryptionkey@0"
I20260812 07:02:03.738400 705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:03.771136 705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:03.776221 705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:03.776552 705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:03.799000 705 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.62:42349
I20260812 07:02:03.799028 820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.62:42349 every 8 connection(s)
I20260812 07:02:03.801505 705 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:03.803369 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 705
I20260812 07:02:03.803586 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-0/wal/instance
I20260812 07:02:03.804324 27528 external_mini_cluster.cc:1428] Setting key e605140d63515d86095d518e287abd4f
I20260812 07:02:03.810750 821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:03.818699 821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:02:03 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518123, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.226.61@KRBTEST.COM; defaulting to no policy
I20260812 07:02:03.846844 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:03.824009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44300 (local address 127.26.226.62:42349)
0812 07:02:03.824840 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:03.824854 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:03.824917 (+ 63us) server_negotiation.cc:407] Connection header received
0812 07:02:03.826453 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:03.826463 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:03.826866 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:03.827181 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:03.828807 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:03.830920 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:03.832606 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:03.833676 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:03.833937 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:03.837693 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:03.837709 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:03.837732 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:03.837780 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:03.842017 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:03.842818 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:03.842828 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:03.842832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:03.843050 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:03.843506 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:03.843513 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:03.843519 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:03.844343 (+ 824us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:03.844542 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:03.845225 (+ 683us) server_negotiation.cc:299] Negotiation successful
0812 07:02:03.845710 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":235,"server-negotiator.queue_time_us":573,"thread_start_us":356,"threads_started":1}
I20260812 07:02:03.846884 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:03.822568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:42349 (local address 127.0.0.1:44300)
0812 07:02:03.823476 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:03.823564 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:03.825878 (+ 2314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:03.827283 (+ 1405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:03.827289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:03.828110 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:03.828536 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:03.828558 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:03.831137 (+ 2579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:03.831141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:03.832434 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:03.832450 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:03.833986 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:03.833992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:03.834166 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:03.834961 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:03.834987 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:03.837369 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:03.842196 (+ 4827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:03.842207 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:03.842219 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:03.842643 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:03.843184 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:03.843187 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:03.843189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:03.843359 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:03.844503 (+ 1144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:03.844510 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:03.844762 (+ 252us) client_negotiation.cc:769] Sending connection context
0812 07:02:03.845002 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 07:02:03.845607 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":1,"thread_start_us":395,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.61@KRBTEST.COM" created.
I20260812 07:02:03.850663 821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:02:03.851961 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:03.851520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52920)
0812 07:02:03.851708 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:03.851839 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 07:02:03.852398 730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
I20260812 07:02:03.854140 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:03.853755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52924)
0812 07:02:03.853913 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:03.853990 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 07:02:03.854359 821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:35037: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab.
Entry for principal kudu/127.26.226.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab.
I20260812 07:02:03.891860 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.61:35037
--webserver_interface=127.26.226.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab
--principal=kudu/127.26.226.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:42349,127.26.226.61:35037,127.26.226.60:45005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:02:03.903740 821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } attempt: 1
I20260812 07:02:03.905069 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:03.904573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52930)
0812 07:02:03.904781 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:03.904902 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 07:02:03.906098 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:03.905738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52936)
0812 07:02:03.905886 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:03.905967 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 07:02:03.906296 821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:35037: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
I20260812 07:02:03.974596 821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } attempt: 2
I20260812 07:02:03.976110 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:03.975372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52940)
0812 07:02:03.975725 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:03.975868 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 07:02:03.977349 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:03.976901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52956)
0812 07:02:03.977071 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:03.977179 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 07:02:03.977576 821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:35037: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
I20260812 07:02:04.068899 821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } attempt: 3
I20260812 07:02:04.070231 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.069701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52962)
0812 07:02:04.069942 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.070059 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 07:02:04.071547 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.070999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52964)
0812 07:02:04.071196 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.071317 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 07:02:04.071861 821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:35037: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
I20260812 07:02:04.215188 821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } attempt: 4
I20260812 07:02:04.216602 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.215921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52966)
0812 07:02:04.216356 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.216448 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254}
I20260812 07:02:04.217890 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.217441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52968)
0812 07:02:04.217647 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.217740 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 07:02:04.218185 821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.61:35037: Network error: Client connection negotiation failed: client connection to 127.26.226.61:35037: connect: Connection refused (error 111)
W20260812 07:02:04.265103 830 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:02:04.265506 830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:04.265606 830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:04.265805 830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:04.273612 830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:04.273720 830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:04.273813 830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:04.273906 830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:04.274019 830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:04.284511 830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:42349,127.26.226.61:35037,127.26.226.60:45005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.61.keytab
--principal=kudu/127.26.226.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.61:35037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.226.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:04.285935 830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:04.287473 830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:04.300705 841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:04.300745 837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:04.300751 836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:04.301942 830 server_base.cc:1034] running on GCE node
I20260812 07:02:04.302865 830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:04.304708 830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:04.305964 830 hybrid_clock.cc:648] HybridClock initialized: now 1786518124305889 us; error 45 us; skew 500 ppm
I20260812 07:02:04.306370 830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:04 dist-test-slave-t3q3 krb5kdc[689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518124, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:04.310174 830 init.cc:377] Logged in from keytab as kudu/127.26.226.61@KRBTEST.COM (short username kudu)
I20260812 07:02:04.312143 830 webserver.cc:460] Webserver started at http://127.26.226.61:46163/ using document root <none> and password file <none>
I20260812 07:02:04.313325 830 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:04.313437 830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:04.313839 830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:04.317276 830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/data/instance:
uuid: "3a413e5daf7e4d7e82663ef3b714aef5"
format_stamp: "Formatted at 2026-08-12 07:02:04 on dist-test-slave-t3q3"
server_key: "7f3d39623b390c7ea35d4bee678d19e6"
server_key_iv: "19058f237989ebbe332294a3fa2cde32"
server_key_version: "encryptionkey@0"
I20260812 07:02:04.318204 830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/wal/instance:
uuid: "3a413e5daf7e4d7e82663ef3b714aef5"
format_stamp: "Formatted at 2026-08-12 07:02:04 on dist-test-slave-t3q3"
server_key: "7f3d39623b390c7ea35d4bee678d19e6"
server_key_iv: "19058f237989ebbe332294a3fa2cde32"
server_key_version: "encryptionkey@0"
I20260812 07:02:04.323876 830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 07:02:04.327720 853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:04.329442 830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:02:04.329679 830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/wal
uuid: "3a413e5daf7e4d7e82663ef3b714aef5"
format_stamp: "Formatted at 2026-08-12 07:02:04 on dist-test-slave-t3q3"
server_key: "7f3d39623b390c7ea35d4bee678d19e6"
server_key_iv: "19058f237989ebbe332294a3fa2cde32"
server_key_version: "encryptionkey@0"
I20260812 07:02:04.329921 830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:04.369544 830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:04.374423 830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:04.374745 830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:04.393878 830 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.61:35037
I20260812 07:02:04.393896 944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.61:35037 every 8 connection(s)
I20260812 07:02:04.395359 830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/data/info.pb
I20260812 07:02:04.399577 945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:04.401436 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 830
I20260812 07:02:04.401650 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-1/wal/instance
I20260812 07:02:04.402127 27528 external_mini_cluster.cc:1428] Setting key 5517134811132654897761c44da733cc
I20260812 07:02:04.406713 945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:02:04 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518124, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:02:04.429596 947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.409550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.26.226.62:42349)
0812 07:02:04.410142 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:04.410150 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:04.411173 (+ 1023us) server_negotiation.cc:407] Connection header received
0812 07:02:04.413281 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:04.413290 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:04.413392 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:04.413539 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:04.415211 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.416390 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.418032 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.418736 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.418844 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:04.423362 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:04.423368 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:04.423371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:04.423420 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:04.426251 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:04.427071 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:04.427083 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:04.427086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:04.427214 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:04.427683 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:04.427691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:04.427694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:04.427985 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:04.428195 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:04.428937 (+ 742us) server_negotiation.cc:299] Negotiation successful
0812 07:02:04.429181 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":285,"threads_started":1}
I20260812 07:02:04.430629 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.410108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:42349 (local address 127.0.0.1:44312)
0812 07:02:04.410952 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.411021 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:04.412959 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:04.413736 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:04.413744 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:04.414345 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:04.414925 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:04.414943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.416588 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.416593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:04.417865 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:04.417878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.419021 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.419031 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:04.419399 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:04.420242 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:04.420280 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:04.423076 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:04.426426 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:04.426437 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:04.426455 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:04.426925 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:04.427345 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:04.427350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:04.427354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:04.427556 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:04.428174 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:04.428182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:04.428556 (+ 374us) client_negotiation.cc:769] Sending connection context
0812 07:02:04.428851 (+ 295us) client_negotiation.cc:240] Negotiation successful
0812 07:02:04.429452 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":364,"threads_started":1}
I20260812 07:02:04.432718 945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.226.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.226.60@KRBTEST.COM" created.
Aug 12 07:02:04 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518124, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 07:02:04.450336 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.433663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52980)
0812 07:02:04.433874 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.433902 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:04.433993 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:04.435225 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:04.435232 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:04.435762 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:04.435904 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:04.435912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.437361 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.437369 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:04.438217 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:04.438228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.439074 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.439079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:04.439319 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:04.439869 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:04.439885 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:04.443032 (+ 3147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:04.447703 (+ 4671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:04.447707 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:04.447711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:04.448163 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:04.448643 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:04.448646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:04.448648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:04.448771 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:04.449583 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:04.449586 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:04.449722 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 07:02:04.449839 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 07:02:04.450050 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":41}
I20260812 07:02:04.450845 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.433911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.26.226.61:35037)
0812 07:02:04.434717 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:04.434723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:04.434751 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:02:04.434795 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:04.434800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:04.435005 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:04.435189 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:04.436114 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.437216 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.438368 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.438920 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.439063 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:04.443286 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:04.443304 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:04.443326 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:04.443379 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:04.447533 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:04.448302 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:04.448314 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:04.448319 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:04.448479 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:04.448888 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:04.448895 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:04.448901 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:04.449450 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:04.449684 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:04.450029 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 07:02:04.450326 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":638,"thread_start_us":443,"threads_started":1}
I20260812 07:02:04.453347 945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:02:04.454651 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.454050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32768)
0812 07:02:04.454253 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.454431 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 07:02:04.454939 855 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
I20260812 07:02:04.456713 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.456245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32776)
0812 07:02:04.456446 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.456552 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 07:02:04.456940 945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:45005: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.226.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab.
Entry for principal kudu/127.26.226.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab.
I20260812 07:02:04.484668 27528 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6poUc4/build/asan/bin/kudu
/tmp/dist-test-task6poUc4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.60:45005
--webserver_interface=127.26.226.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab
--principal=kudu/127.26.226.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.226.20:45193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.226.62:42349,127.26.226.61:35037,127.26.226.60:45005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:02:04.506342 945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } attempt: 1
I20260812 07:02:04.507702 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.507198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32792)
0812 07:02:04.507416 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.507538 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 07:02:04.508852 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.508473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32794)
0812 07:02:04.508654 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.508742 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 07:02:04.509079 945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:45005: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
I20260812 07:02:04.517450 821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } attempt: 5
Aug 12 07:02:04 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518123, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 07:02:04.531021 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.518249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52994 (local address 127.26.226.61:35037)
0812 07:02:04.518449 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:04.518456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:04.518559 (+ 103us) server_negotiation.cc:407] Connection header received
0812 07:02:04.518786 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:04.518798 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:04.518877 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:04.518987 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:04.519784 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.520550 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.521727 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.522179 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.522326 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:04.525765 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:04.525770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:04.525773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:04.525805 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:04.528488 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:04.529282 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:04.529286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:04.529288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:04.529405 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:04.529811 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:04.529814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:04.529816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:04.530110 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:04.530224 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:04.530649 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 07:02:04.530811 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 07:02:04.531210 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.518239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:52994)
0812 07:02:04.518450 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.518471 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:04.518654 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:04.519020 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:04.519025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:04.519442 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:04.519634 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:04.519644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.520689 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.520694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:04.521576 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:04.521588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:04.522330 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:04.522335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:04.522541 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:04.523181 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:04.523198 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:04.525578 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:04.528622 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:04.528627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:04.528632 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:04.529170 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:04.529521 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:04.529528 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:04.529535 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:04.529704 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:04.530227 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:04.530232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:04.530405 (+ 173us) client_negotiation.cc:769] Sending connection context
0812 07:02:04.530556 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 07:02:04.530858 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 07:02:04.532281 821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:02:04.533336 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.532861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32800)
0812 07:02:04.533056 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.533164 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:02:04.534420 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.534010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32802)
0812 07:02:04.534188 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.534269 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 07:02:04.534690 821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:45005: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
I20260812 07:02:04.577322 945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } attempt: 2
I20260812 07:02:04.578569 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.578078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32808)
0812 07:02:04.578288 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.578395 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 07:02:04.579785 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.579430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32810)
0812 07:02:04.579591 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.579669 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 07:02:04.580145 945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:45005: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
I20260812 07:02:04.585949 821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } attempt: 1
I20260812 07:02:04.587215 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.586655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32816)
0812 07:02:04.586874 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.586986 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 07:02:04.588460 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.587983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32826)
0812 07:02:04.588194 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.588299 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 07:02:04.588727 821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:45005: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
I20260812 07:02:04.657027 821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } attempt: 2
I20260812 07:02:04.658571 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.657849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32830)
0812 07:02:04.658147 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.658295 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260812 07:02:04.659813 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.659370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32838)
0812 07:02:04.659544 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.659665 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 07:02:04.660117 821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:45005: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
I20260812 07:02:04.671397 945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } attempt: 3
I20260812 07:02:04.672652 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.672179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32844)
0812 07:02:04.672373 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.672479 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:02:04.673919 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.673460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32846)
0812 07:02:04.673717 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.673806 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260812 07:02:04.674232 945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:45005: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
I20260812 07:02:04.766359 821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } attempt: 3
I20260812 07:02:04.767781 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.767189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32860)
0812 07:02:04.767432 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.767564 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 07:02:04.769024 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.768574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32876)
0812 07:02:04.768768 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.768874 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 07:02:04.769265 821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:45005: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
I20260812 07:02:04.817484 945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } attempt: 4
I20260812 07:02:04.818727 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.818238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32888)
0812 07:02:04.818467 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.818560 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 07:02:04.819775 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.819423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32892)
0812 07:02:04.819582 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.819661 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 07:02:04.819999 945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:45005: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
W20260812 07:02:04.863139 956 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:02:04.863570 956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:04.863705 956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:04.863850 956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:04.872098 956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:04.872246 956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:04.872351 956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:04.872444 956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:04.872556 956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:04.883203 956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.226.20:45193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.226.62:42349,127.26.226.61:35037,127.26.226.60:45005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/krb5kdc/kudu_127.26.226.60.keytab
--principal=kudu/127.26.226.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.226.60:45005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.226.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:04.884691 956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:04.886125 956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:04.898729 964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:04.898869 963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:04.898752 970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:04.900647 956 server_base.cc:1034] running on GCE node
I20260812 07:02:04.901413 956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:04.902761 956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:04.904011 956 hybrid_clock.cc:648] HybridClock initialized: now 1786518124903913 us; error 60 us; skew 500 ppm
I20260812 07:02:04.904404 956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:04 dist-test-slave-t3q3 krb5kdc[689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518124, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:04.907678 956 init.cc:377] Logged in from keytab as kudu/127.26.226.60@KRBTEST.COM (short username kudu)
I20260812 07:02:04.909163 956 webserver.cc:460] Webserver started at http://127.26.226.60:38493/ using document root <none> and password file <none>
I20260812 07:02:04.909825 956 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:04.909915 956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:04.910195 956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:04.913244 956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/data/instance:
uuid: "2941f6e0bdd745a49ab25e67c5f7d55f"
format_stamp: "Formatted at 2026-08-12 07:02:04 on dist-test-slave-t3q3"
server_key: "72d2d9e600287ec24bca20d252d3a7ab"
server_key_iv: "5e69b2a774f07fa613043542c384cf41"
server_key_version: "encryptionkey@0"
I20260812 07:02:04.913993 956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/wal/instance:
uuid: "2941f6e0bdd745a49ab25e67c5f7d55f"
format_stamp: "Formatted at 2026-08-12 07:02:04 on dist-test-slave-t3q3"
server_key: "72d2d9e600287ec24bca20d252d3a7ab"
server_key_iv: "5e69b2a774f07fa613043542c384cf41"
server_key_version: "encryptionkey@0"
I20260812 07:02:04.919063 956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260812 07:02:04.922617 981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:04.923923 956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:02:04.924109 956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/data,/tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/wal
uuid: "2941f6e0bdd745a49ab25e67c5f7d55f"
format_stamp: "Formatted at 2026-08-12 07:02:04 on dist-test-slave-t3q3"
server_key: "72d2d9e600287ec24bca20d252d3a7ab"
server_key_iv: "5e69b2a774f07fa613043542c384cf41"
server_key_version: "encryptionkey@0"
I20260812 07:02:04.924331 956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:04.946482 821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } attempt: 4
I20260812 07:02:04.947685 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.947184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32896)
0812 07:02:04.947397 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.947509 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:02:04.948818 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.948429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32910)
0812 07:02:04.948582 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:04.948669 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 07:02:04.949070 821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.226.60:45005: Network error: Client connection negotiation failed: client connection to 127.26.226.60:45005: connect: Connection refused (error 111)
I20260812 07:02:04.960978 956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:04.965629 956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:04.965925 956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:04.985930 956 rpc_server.cc:307] RPC server started. Bound to: 127.26.226.60:45005
I20260812 07:02:04.985951 1071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.226.60:45005 every 8 connection(s)
I20260812 07:02:04.987408 956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/data/info.pb
I20260812 07:02:04.991250 1075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:04.994272 27528 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6poUc4/build/asan/bin/kudu as pid 956
I20260812 07:02:04.994485 27528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6poUc4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517734245872-27528-0/minicluster-data/master-2/wal/instance
I20260812 07:02:04.995048 27528 external_mini_cluster.cc:1428] Setting key 58f8f3cc2a0254e861e00af878f98d81
I20260812 07:02:04.999683 1075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:02:05 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
Aug 12 07:02:05 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518124, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:02:05.019914 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.002190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44324 (local address 127.26.226.62:42349)
0812 07:02:05.002717 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.002725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.003412 (+ 687us) server_negotiation.cc:407] Connection header received
0812 07:02:05.003599 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.003605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.003680 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.003795 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.004986 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.005996 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.008110 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.008831 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.008977 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.012937 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.012944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.012950 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.013013 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.016441 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.017284 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.017290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.017294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.017441 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.017887 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.017892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.017895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.018216 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.018414 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.019104 (+ 690us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.019338 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":271,"threads_started":1}
I20260812 07:02:05.020831 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:04.997042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32912 (local address 127.26.226.60:45005)
0812 07:02:04.997927 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:04.997939 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:04.997982 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:02:04.999638 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:04.999661 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.000157 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.000547 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.002083 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.003924 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.005471 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.006446 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.006685 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.012259 (+ 5574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.012268 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.012281 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.012319 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.016050 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.016955 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.016962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.016964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.017151 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.017617 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.017621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.017623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.018314 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.018598 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.019002 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.019487 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":658,"thread_start_us":409,"threads_started":1}
I20260812 07:02:05.020851 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.002207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.62:42349 (local address 127.0.0.1:44324)
0812 07:02:05.003244 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:05.003301 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:05.003457 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:05.003856 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:05.003862 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:05.004567 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:05.004821 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:05.004830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.006184 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.006190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:05.007886 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:05.007902 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.009176 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.009187 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:05.009419 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.010097 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:05.010149 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:05.012716 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:05.016579 (+ 3863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:05.016592 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:05.016608 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:05.017150 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:05.017565 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:05.017572 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:05.017577 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:05.017757 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:05.018368 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:05.018374 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:05.018663 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 07:02:05.018992 (+ 329us) client_negotiation.cc:240] Negotiation successful
0812 07:02:05.019717 (+ 725us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":181,"thread_start_us":535,"threads_started":1}
I20260812 07:02:05.022506 1075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:02:05 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518124, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
Aug 12 07:02:05 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.62@KRBTEST.COM
I20260812 07:02:05.038143 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.023433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.61:35037 (local address 127.0.0.1:53010)
0812 07:02:05.023799 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:05.023815 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:05.023958 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:05.024548 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:05.024553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:05.025092 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:05.025298 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:05.025312 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.026456 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.026460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:05.027586 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:05.027603 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.028440 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.028446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:05.028693 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.029324 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:05.029343 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:05.031777 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:05.035485 (+ 3708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:05.035491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:05.035495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:05.035985 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:05.036446 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:05.036451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:05.036455 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:05.036605 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:05.037148 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:05.037153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:05.037317 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 07:02:05.037472 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 07:02:05.037770 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:02:05.038174 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.023534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53010 (local address 127.26.226.61:35037)
0812 07:02:05.023734 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.023740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.023925 (+ 185us) server_negotiation.cc:407] Connection header received
0812 07:02:05.024135 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.024141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.024232 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.024341 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.025513 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.026306 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.027765 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.028284 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.028388 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.031979 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.031985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.031988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.032020 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.035355 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.036167 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.036179 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.036182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.036318 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.036720 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.036726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.036729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.037007 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.037162 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.037567 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.037786 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:02:05.039247 1075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:02:05.040040 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.024191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44340 (local address 127.26.226.62:42349)
0812 07:02:05.024353 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.024357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.024373 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:02:05.024517 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.024521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.024603 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.024679 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.026190 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.026956 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.028619 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.029365 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.029538 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.034050 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.034056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.034061 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.034108 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.036919 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.037650 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.037654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.037657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.037786 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.038184 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.038188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.038190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.038584 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.038851 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.039486 (+ 635us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.039677 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 07:02:05 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518124, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.60@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
Aug 12 07:02:05 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.226.61@KRBTEST.COM
I20260812 07:02:05.052582 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.040159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32928 (local address 127.26.226.60:45005)
0812 07:02:05.040457 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.040463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.040482 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:02:05.040531 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.040535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.040612 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.040714 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.041455 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.042214 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.043368 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.043785 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.043922 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.047191 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.047196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.047199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.047234 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.049869 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.050574 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.050579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.050583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.050715 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.051079 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.051085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.051088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.051364 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.051520 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.051897 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.052172 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":190}
I20260812 07:02:05.052702 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.040026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32928)
0812 07:02:05.040229 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:05.040246 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:05.040338 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:05.040737 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:05.040742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:05.041167 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:05.041320 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:05.041329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.042346 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.042353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:05.043226 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:05.043238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.043930 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.043935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:05.044237 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.044823 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:05.044841 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:05.047006 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:05.050013 (+ 3007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:05.050018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:05.050022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:05.050447 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:05.050827 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:05.050833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:05.050837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:05.050971 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:05.051486 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:05.051491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:05.051641 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 07:02:05.051802 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 07:02:05.052323 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":227}
I20260812 07:02:05.057096 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.042008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53024 (local address 127.26.226.61:35037)
0812 07:02:05.042174 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.042179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.042194 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:02:05.042246 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.042249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.042299 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.042403 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.043624 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.044433 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.045837 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.046409 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.046546 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.051227 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.051233 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.051236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.051269 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.054248 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.054991 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.054997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.055001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.055177 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.055564 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.055570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.055573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.055921 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.056229 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.056498 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.056736 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:02:05.058720 1075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f: Bootstrap starting.
I20260812 07:02:05.062037 27528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:02:05.062654 1075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:05.063994 1075 log.cc:824] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:05.067324 1075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f: No bootstrap required, opened a new log
I20260812 07:02:05.065367 784 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ce0533c54e50486aa1584ca023251f33"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44340
I20260812 07:02:05.074685 1075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } }
I20260812 07:02:05.075022 1075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2941f6e0bdd745a49ab25e67c5f7d55f, State: Initialized, Role: FOLLOWER
I20260812 07:02:05.075554 1075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } }
I20260812 07:02:05.076588 1087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } } }
I20260812 07:02:05.076988 1087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:05.077646 1075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:02:05.087493 1103 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:02:05.087643 1103 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:02:05.085450 956 master_runner.cc:186] Error getting master registration for 127.26.226.62:42349: OK, instance_id { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" instance_seqno: 1786518123791254 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:02:05.097292 956 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:02:05.119246 945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } attempt: 5
Aug 12 07:02:05 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518124, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.61@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
I20260812 07:02:05.137485 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.120121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32936)
0812 07:02:05.120360 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:05.120382 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:05.120536 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:05.121030 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:05.121035 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:05.121609 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:05.121829 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:05.121842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.123386 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.123393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:05.124861 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:05.124878 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.125961 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.125966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:05.126221 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.126964 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:05.126987 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:05.130003 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:05.134297 (+ 4294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:05.134305 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:05.134311 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:05.134909 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:05.135401 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:05.135408 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:05.135414 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:05.135581 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:05.136414 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:05.136419 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:05.136590 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 07:02:05.136733 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 07:02:05.137057 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 07:02:05.137538 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.120381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.26.226.60:45005)
0812 07:02:05.120633 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.120639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.120663 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:02:05.120740 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.120748 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.120855 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.120988 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.122018 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.123217 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.125026 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.125787 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.125933 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.130234 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.130243 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.130248 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.130294 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.134135 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.135073 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.135083 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.135087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.135250 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.135719 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.135725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.135731 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.136281 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.136476 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.136905 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.137160 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 07:02:05.143911 945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5: Bootstrap starting.
I20260812 07:02:05.147677 945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:05.149163 945 log.cc:824] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:05.151927 945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5: No bootstrap required, opened a new log
I20260812 07:02:05.163338 945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } }
I20260812 07:02:05.163784 945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a413e5daf7e4d7e82663ef3b714aef5, State: Initialized, Role: FOLLOWER
I20260812 07:02:05.164394 945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } }
I20260812 07:02:05.165482 1106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } } }
I20260812 07:02:05.165902 1106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:05.166736 945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:02:05.179268 1126 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:02:05.179498 1126 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:02:05.176470 830 master_runner.cc:186] Error getting master registration for 127.26.226.62:42349: OK, instance_id { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" instance_seqno: 1786518123791254 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:02:05.189077 830 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 07:02:05.218786 990 tablet.cc:2369] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:05.226325 821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } attempt: 5
Aug 12 07:02:05 dist-test-slave-t3q3 krb5kdc[689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518123, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.226.62@KRBTEST.COM for kudu/127.26.226.60@KRBTEST.COM
I20260812 07:02:05.243574 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.227218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.26.226.60:45005)
0812 07:02:05.227868 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.227873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.227889 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:02:05.227985 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.227990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.228099 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.228232 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.229261 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.230047 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.232008 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.232522 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.232680 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.236943 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.236953 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.236956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.236995 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.240728 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.241406 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.241415 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.241420 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.241594 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.242023 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.242030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.242034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.242408 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.242564 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.242967 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.243219 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534}
I20260812 07:02:05.243589 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.227133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.226.60:45005 (local address 127.0.0.1:32950)
0812 07:02:05.227428 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:05.227447 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:05.227596 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:05.228275 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:05.228284 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:05.228893 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:05.229114 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:05.229129 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.230210 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.230217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:05.231855 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:05.231874 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.232695 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.232701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:05.232903 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.233591 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:05.233615 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:05.236741 (+ 3126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:05.240865 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:05.240870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:05.240875 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:05.241263 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:05.241717 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:05.241722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:05.241726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:05.241895 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:05.242538 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:05.242543 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:05.242718 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 07:02:05.242870 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 07:02:05.243165 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260812 07:02:05.249049 821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33: Bootstrap starting.
I20260812 07:02:05.252580 821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:05.254036 821 log.cc:824] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:05.257767 821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33: No bootstrap required, opened a new log
I20260812 07:02:05.265545 821 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } }
I20260812 07:02:05.266001 821 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0533c54e50486aa1584ca023251f33, State: Initialized, Role: FOLLOWER
I20260812 07:02:05.266579 821 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } }
I20260812 07:02:05.268302 1130 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } } }
I20260812 07:02:05.269049 1130 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:05.269564 821 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:05.269622 784 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 07:02:05.269848 784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:05.273974 784 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } }
I20260812 07:02:05.277400 895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce0533c54e50486aa1584ca023251f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3a413e5daf7e4d7e82663ef3b714aef5"
I20260812 07:02:05.277940 895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:05.278081 1029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce0533c54e50486aa1584ca023251f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f"
I20260812 07:02:05.278653 1029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [term 0 FOLLOWER]: Advancing to term 1
W20260812 07:02:05.279837 737 tablet.cc:2369] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:05.281553 895 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0533c54e50486aa1584ca023251f33 in term 1.
I20260812 07:02:05.282251 730 leader_election.cc:302] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a413e5daf7e4d7e82663ef3b714aef5, ce0533c54e50486aa1584ca023251f33; no voters:
I20260812 07:02:05.284422 1029 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0533c54e50486aa1584ca023251f33 in term 1.
I20260812 07:02:05.285415 784 leader_election.cc:288] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [CANDIDATE]: Term 1 election: Requested vote from peers 3a413e5daf7e4d7e82663ef3b714aef5 (127.26.226.61:35037), 2941f6e0bdd745a49ab25e67c5f7d55f (127.26.226.60:45005)
I20260812 07:02:05.293087 1130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:05.301955 1130 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [term 1 LEADER]: Becoming Leader. State: Replica: ce0533c54e50486aa1584ca023251f33, State: Running, Role: LEADER
I20260812 07:02:05.303233 1130 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } }
I20260812 07:02:05.325582 1137 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce0533c54e50486aa1584ca023251f33. Latest consensus state: current_term: 1 leader_uuid: "ce0533c54e50486aa1584ca023251f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } } }
I20260812 07:02:05.326279 1137 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:05.340852 1149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260812 07:02:05.377635 864 tablet.cc:2369] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:05.387363 1149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:05.395664 1167 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:02:05.415306 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.315684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53030 (local address 127.26.226.61:35037)
0812 07:02:05.315905 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.315911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.335865 (+ 19954us) server_negotiation.cc:407] Connection header received
0812 07:02:05.336231 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.336240 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.336359 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.336450 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.354704 (+ 18254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.356200 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.374706 (+ 18506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.375729 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.375893 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.384830 (+ 8937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.384837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.384842 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.384893 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.388827 (+ 3934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.412290 (+ 23463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.412296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.412300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.412517 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.413070 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.413077 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.413085 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.413598 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.413859 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.414589 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.414889 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:02:05.423329 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.332760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.26.226.60:45005)
0812 07:02:05.333039 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.333048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.345603 (+ 12555us) server_negotiation.cc:407] Connection header received
0812 07:02:05.346349 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.346359 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.346477 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.346581 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.366870 (+ 20289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.368802 (+ 1932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.384797 (+ 15995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.385586 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.386024 (+ 438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.406334 (+ 20310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.406345 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.406351 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.406401 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.410626 (+ 4225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.411621 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.411627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.411631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.411794 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.419837 (+ 8043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.419851 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.419856 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.420344 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.420648 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.422620 (+ 1972us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.422922 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:02:05.424911 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:05.314558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.26.226.62:42349)
0812 07:02:05.314739 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.314748 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.330110 (+ 15362us) server_negotiation.cc:407] Connection header received
0812 07:02:05.330478 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.330486 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.330615 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.330712 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.348738 (+ 18026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.350245 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.369918 (+ 19673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:05.370986 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:05.371136 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:05.412288 (+ 41152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:05.412296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:05.412302 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:05.412350 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:05.415963 (+ 3613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.419053 (+ 3090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.419059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.419063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.419218 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:05.421182 (+ 1964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:05.421190 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:05.421195 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:05.421613 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:05.421970 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:05.424271 (+ 2301us) server_negotiation.cc:299] Negotiation successful
0812 07:02:05.424548 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 07:02:05.441380 1029 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [term 1 FOLLOWER]: Refusing update from remote peer ce0533c54e50486aa1584ca023251f33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:05.441452 895 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [term 1 FOLLOWER]: Refusing update from remote peer ce0533c54e50486aa1584ca023251f33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:05.442526 1137 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:05.443465 1130 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:05.472631 1106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce0533c54e50486aa1584ca023251f33. Latest consensus state: current_term: 1 leader_uuid: "ce0533c54e50486aa1584ca023251f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } } }
I20260812 07:02:05.473001 1106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:05.496745 1106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce0533c54e50486aa1584ca023251f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } } }
I20260812 07:02:05.497082 1106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:05.499504 1087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce0533c54e50486aa1584ca023251f33. Latest consensus state: current_term: 1 leader_uuid: "ce0533c54e50486aa1584ca023251f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } } }
I20260812 07:02:05.499953 1087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:05.503531 1137 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce0533c54e50486aa1584ca023251f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } } }
I20260812 07:02:05.503856 1137 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:05.504174 1137 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce0533c54e50486aa1584ca023251f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } } }
I20260812 07:02:05.504431 1137 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:05.505316 1149 catalog_manager.cc:1347] Generated new cluster ID: a66b0bc773054abcba56b5fe63ca74a4
I20260812 07:02:05.505419 1149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:05.518424 1087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce0533c54e50486aa1584ca023251f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0533c54e50486aa1584ca023251f33" member_type: VOTER last_known_addr { host: "127.26.226.62" port: 42349 } } peers { permanent_uuid: "3a413e5daf7e4d7e82663ef3b714aef5" member_type: VOTER last_known_addr { host: "127.26.226.61" port: 35037 } } peers { permanent_uuid: "2941f6e0bdd745a49ab25e67c5f7d55f" member_type: VOTER last_known_addr { host: "127.26.226.60" port: 45005 } } }
I20260812 07:02:05.518750 1087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:02:05.537355 1149 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:05.543663 1149 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:05.570209 1149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce0533c54e50486aa1584ca023251f33: Generated new TSK 0
I20260812 07:02:05.571566 1149 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:02:05.704551 1082 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.688646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44370 (local address 127.26.226.62:42349)
0812 07:02:05.688894 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.688902 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.702458 (+ 13556us) server_negotiation.cc:407] Connection header received
0812 07:02:05.702779 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.702786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.702877 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.702959 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.704272 (+ 1313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44370: BlockingRecv error: recv got EOF from 127.0.0.1:44370 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 07:02:05.706223 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.689083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53046 (local address 127.26.226.61:35037)
0812 07:02:05.689320 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.689326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.702197 (+ 12871us) server_negotiation.cc:407] Connection header received
0812 07:02:05.703864 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.703871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.703966 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.704051 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.705956 (+ 1905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53046: BlockingRecv error: recv got EOF from 127.0.0.1:53046 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260812 07:02:05.707049 1079 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.702828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32976 (local address 127.26.226.60:45005)
0812 07:02:05.703021 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.703027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.705405 (+ 2378us) server_negotiation.cc:407] Connection header received
0812 07:02:05.705665 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.705671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.705759 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.705833 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.706782 (+ 949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32976: BlockingRecv error: recv got EOF from 127.0.0.1:32976 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 07:02:05.718276 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.712678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53056 (local address 127.26.226.61:35037)
0812 07:02:05.712875 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.712881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.716567 (+ 3686us) server_negotiation.cc:407] Connection header received
0812 07:02:05.716787 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.716793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.716961 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.717092 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.717918 (+ 826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53056: BlockingRecv error: recv got EOF from 127.0.0.1:53056 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 07:02:05.726100 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.722811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53060 (local address 127.26.226.61:35037)
0812 07:02:05.722981 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.722986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.724610 (+ 1624us) server_negotiation.cc:407] Connection header received
0812 07:02:05.724784 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.724790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.724894 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.725018 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.725858 (+ 840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53060: BlockingRecv error: recv got EOF from 127.0.0.1:53060 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 07:02:05.734349 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.731450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.26.226.61:35037)
0812 07:02:05.731619 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.731625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.732691 (+ 1066us) server_negotiation.cc:407] Connection header received
0812 07:02:05.732893 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.732900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.733044 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.733179 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.734083 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53062: BlockingRecv error: recv got EOF from 127.0.0.1:53062 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 07:02:05.749169 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.746709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.26.226.61:35037)
0812 07:02:05.746947 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.746954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.747563 (+ 609us) server_negotiation.cc:407] Connection header received
0812 07:02:05.747750 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.747758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.747860 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.747980 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.748905 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53068: BlockingRecv error: recv got EOF from 127.0.0.1:53068 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
W20260812 07:02:05.762876 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.760571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53082 (local address 127.26.226.61:35037)
0812 07:02:05.760741 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.760747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.761313 (+ 566us) server_negotiation.cc:407] Connection header received
0812 07:02:05.761497 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.761511 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.761613 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.761730 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.762611 (+ 881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53082: BlockingRecv error: recv got EOF from 127.0.0.1:53082 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 07:02:05.785022 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.782553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.26.226.61:35037)
0812 07:02:05.782824 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.782831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.783193 (+ 362us) server_negotiation.cc:407] Connection header received
0812 07:02:05.783416 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.783421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.783499 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.783794 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.784743 (+ 949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53090: BlockingRecv error: recv got EOF from 127.0.0.1:53090 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260812 07:02:05.823851 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.821734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53104 (local address 127.26.226.61:35037)
0812 07:02:05.821949 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.821956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.822375 (+ 419us) server_negotiation.cc:407] Connection header received
0812 07:02:05.822547 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.822553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.822667 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.822793 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.823585 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53104: BlockingRecv error: recv got EOF from 127.0.0.1:53104 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260812 07:02:05.894007 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:05.891927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53120 (local address 127.26.226.61:35037)
0812 07:02:05.892200 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:05.892208 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:05.892541 (+ 333us) server_negotiation.cc:407] Connection header received
0812 07:02:05.892815 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:05.892819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:05.892895 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:05.893026 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:05.893775 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53120: BlockingRecv error: recv got EOF from 127.0.0.1:53120 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260812 07:02:06.030949 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:06.028928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53136 (local address 127.26.226.61:35037)
0812 07:02:06.029203 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:06.029214 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:06.029563 (+ 349us) server_negotiation.cc:407] Connection header received
0812 07:02:06.029726 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:06.029730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:06.029807 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:06.029901 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:06.030671 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53136: BlockingRecv error: recv got EOF from 127.0.0.1:53136 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 07:02:06.089695 1103 catalog_manager.cc:1259] Loaded cluster ID: a66b0bc773054abcba56b5fe63ca74a4
I20260812 07:02:06.089800 1103 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f: loading cluster ID for follower catalog manager: success
I20260812 07:02:06.092495 1103 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f: acquiring CA information for follower catalog manager: success
I20260812 07:02:06.094223 1103 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2941f6e0bdd745a49ab25e67c5f7d55f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:02:06.181774 1126 catalog_manager.cc:1259] Loaded cluster ID: a66b0bc773054abcba56b5fe63ca74a4
I20260812 07:02:06.181918 1126 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5: loading cluster ID for follower catalog manager: success
I20260812 07:02:06.184480 1126 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5: acquiring CA information for follower catalog manager: success
I20260812 07:02:06.186292 1126 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3a413e5daf7e4d7e82663ef3b714aef5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:02:06.300007 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:06.294046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53148 (local address 127.26.226.61:35037)
0812 07:02:06.294287 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:06.294294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:06.294460 (+ 166us) server_negotiation.cc:407] Connection header received
0812 07:02:06.294731 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:06.294735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:06.294895 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:06.295032 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:06.295516 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:06.296302 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:06.297686 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:06.298097 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:06.298242 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:06.298768 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:06.299321 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:06.299502 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:06.299619 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 07:02:06.299710 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 07:02:06.312315 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:06.306542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32990 (local address 127.26.226.60:45005)
0812 07:02:06.307025 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:06.307030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:06.307049 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:02:06.307178 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:06.307182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:06.307340 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:06.307491 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:06.307888 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:06.308696 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:06.310112 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:06.310518 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:06.310615 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:06.311161 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:06.311694 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:06.311815 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:06.311921 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 07:02:06.311994 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":267,"threads_started":1}
I20260812 07:02:06.326529 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:06.318820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53156 (local address 127.26.226.61:35037)
0812 07:02:06.319069 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:06.319078 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:06.320164 (+ 1086us) server_negotiation.cc:407] Connection header received
0812 07:02:06.320484 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:06.320490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:06.320593 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:06.320673 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:06.321542 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:06.322384 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:06.324490 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:06.324956 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:06.325108 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:06.325723 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:06.325863 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:06.325962 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:06.326121 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 07:02:06.326220 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 07:02:06.327435 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:06.318758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.26.226.62:42349)
0812 07:02:06.319551 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:06.319558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:06.320123 (+ 565us) server_negotiation.cc:407] Connection header received
0812 07:02:06.320418 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:06.320428 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:06.320716 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:06.320921 (+ 205us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:06.321632 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:06.322621 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:06.324557 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:06.325211 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:06.325362 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:06.325943 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:06.326436 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:06.326556 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:06.327045 (+ 489us) server_negotiation.cc:299] Negotiation successful
0812 07:02:06.327137 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":501,"threads_started":1}
I20260812 07:02:06.327678 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:06.320362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33006 (local address 127.26.226.60:45005)
0812 07:02:06.320533 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:06.320539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:06.321125 (+ 586us) server_negotiation.cc:407] Connection header received
0812 07:02:06.321342 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:06.321347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:06.321435 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:06.321529 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:06.322214 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:06.323235 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:06.325449 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:06.326071 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:06.326180 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:06.326870 (+ 690us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:06.326988 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:06.327053 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:06.327303 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 07:02:06.327385 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:02:06.333173 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 705
I20260812 07:02:06.334331 816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:06.446703 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 705
I20260812 07:02:06.482853 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 830
I20260812 07:02:06.484143 937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:06.594913 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 830
I20260812 07:02:06.623355 27528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6poUc4/build/asan/bin/kudu pid 956
I20260812 07:02:06.624603 1062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:06.737483 27528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6poUc4/build/asan/bin/kudu with pid 956
2026-08-12T07:02:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5262 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10453 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (392406 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST